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(57) The present invention provides methods and 
apparatus for coding and/or encoding a bit stream com- 
prising a representation of a sequence of n-dimensionai 
data structures or matrices, in which n is typically 2. A 
part of the data of one data structure of the sequence 
can be mappable within predefined similarity critreria to 
a part of the data of a another data structure of the se- 
quence. The similarity criteria may include, for instance, 
a spatial or temporal shift of the data within an image of 
a video signal such as is used in motion estimation and/ 
or compensation of moving pictures, e.g. video images 
as well as coders and encoders for coding such a bit- 
stream, e.g. a video signal for use in motion compensa- 
tion and/or estimation. The data structures are typically 



video frames and a part of the data structure may be a 
block of data within a frame. The shift may be any suit- 
able shift such as a linear translation at any suitable an- 
gle, a rotation of the data or change of size such as 
zooming between a part of the data in one data structure 
and a part of the data in another data structure of the 
sequence. The data structures are typically sequential 
frames of a video information stream. 

Digital filtering methods and apparatus are provided 
for application to a reference or other frame of data to 
generate one or more subbands of a set of subbands of 
an overcomplete representation of the frame by calcu- 
lations at single rate. These filters may be implemented 
as a separate filter module or may be implemented as 
software running on a microprocessor, for example. 



CM 
< 

CO 
O) 



CM 



Q. 

LU 




Fig. 4 



Printed by Jouve. 75001 PARIS (FR) 



EP1 294 196 A2 



Description 

[0001] The present invention relates to methods of coding and encoding a bit stream comprising a representation 
of a sequence of n-dimensional data structures or matrices, in which n is typically 2. A part of the data of one data 
structure of the sequence can be mappable within predefined similarity critreria to a part of the data of another data 
structure of the sequence, as is used in motion compensation and/or estimation of moving pictures, e.g. video images. 
The present invention also includes decoders and encoders for coding such a bitstream, e.g. a video signal for use in 
motion estimation and/or compensation as well as filter modules for digital filtering of such bit streams and computer 
program products for executing such filtering as well as subband coding methods. The present invention is particularly 
relevant to in-band motion estimation/motion compensation of video images. 

BACKGROUND OF THE INVENTION 

[0002] Wavelet-based coding has been generally accepted as the most efficient technique for still-picture compres- 
sion. Wavelet transform schemes are described in detail in "Wavelets and Subbands", by Abbate, DeCusatis and Das, 
Birkhauser press, 2002. The insertion of discrete wavelet transforms (DWT) in the new JPEG-2000 coding standard 
led to increased coding efficiency in comparison to previous standards in this area, and additionally, a number of 
interesting features including quality and resolution scalability, stemming from the multiresolution nature of the trans- 
form are provided. In the video-coding arena, although such scalability features (along with temporal scalability) are 
highly desired in a number of applications (like video streaming and multimedia over networks), wavelets are employed 
only for the texture coding in the MPEG-4 standard at present. To address scalability, the MPEG-4 standard adopts 
the multiresolution DCT approach within a hybrid coding structure, which performs relatively poorly in the complexity 
versus coding efficiency sense in comparison to wavelets. For these reasons, many authors have begun to explore 
wavelet-based scalable video-coding schemes. Until recently, the research efforts were mainly directed towards the 
use of 3-D wavelet decompositions for each input group of frames (GOF) in order to remove the spatial and the temporal 
redundancies in the video stream. This work was pioneered mainly by Karisson and Vetterli [1], Lewis and Knowles 
[2], and more recently by Ohm [3] and Taubman and Zakhor [4] who introduced 3-D decompositions coupled with 
motion estimation (ME) and motion compensation (MC). More recent algorithms proposed by Kim, Xiong and Pearlman 
[5) and Bottreau et al [6] support ail types of scalability (spatial, quality and temporal by using 3-D versions of the SPIHT 
algorithm [7] and hierarchical spatial -domain techniques for block-based ME and MC. A wavelet decomposition using 
a short filter-pair like the Haar transform is performed in the temporal direction to remove the redundancies between 
successive residual frames. Furthermore, a 2-D wavelet decomposition of the motion compensated sequence (i.e. the 
residual frames) is performed to reduce spatial redundancies and to compact the energy in the lower-frequency sub- 
bands (using classical filters from still-image coding, such as the 9/7 filter-pair). Quality scalability can be obtained with 
this type of algorithms by coding the three-dimensional transform-domain coefficients using the 3-D extensions [5] of 
the classical 2-D embedded zerotree-based [7] or block-based wavelet image coders [8][9]. Spatial scalability can be 
achieved only if the motion compensation is performed in a level-by-level manner. In addition, temporal scalability is 
inherent to such schemes, since in a multilevel temporal decomposition each resolution reconstructs to a dyadically- 
reduced frame-rate for the decoded sequence. In conclusion, these schemes algorithmically satisfy the scalability 
issues, and moreover, they provide good coding performance. Nevertheless, their limitation comes from the implemen- 
tation point of view because they require a large memory budget for the 3-D transform-application to each GOF, and 
they distribute almost equally the computational load between the encoder and the decoder, thus making the decoder 
implementation relatively complex. In addition, the complete codec delay is also increased since the decoder can 
receive compressed data only after the full 3-D transform is completed in the current GOF of the encoder. Thus they 
are insufficient for bi-directional communications and for applications where power dissipation and memory are major 
cost issues, i.e. for portable systems. Other approaches for scalable wavelet video coding which try to reduce the 
implementation complexity and the system delay follow the classical MPEG-alike hybrid coding-structure, where the 
ME/MC is also performed in the spatial domain and the DCT transform is replaced with a wavelet transform. Typical 
examples of such systems are described in [10] and [11]. Although scalability in quality can be achieved by embedded 
wavelet coding [7] [8][9], the main drawback of such techniques is that they fail to take advantage of the inherent 
multiresolution structure of the wavelet transform to provide drift-free spatial scalability. In addition, there is an inverse 
transform in the coding loop, resulting to two transform-applications (one forward and one inverse) per frame and per 
spatial resolution. This may also lead to large codec delays, since no parallelism is possible and each wavelet transform 
must be applied to the complete frame. More recent research efforts tie the classic hybrid coding-structure with motion 
estimation and compensation techniques in the wavelet domain, leading to the so called in-band ME/MC class of 
wavelet video codecs [12][13][14] [16]. This class of codecs presents a conceptually more appealing approach since 
the multiresolution features of the transform can be used so as to provide an inherent spatial and quality scalability 
similar to wavelet-based coding of still images. Hence, if motion compensation is performed in a level-by-level manner 
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in the wavelet subbands, a decoder can decode without drift a video sequence with the horizontal and vertical frame- 
dimensions having half or quarter-size, since the same information as the encoder is utilized. In addition, the complexity 
is reduced, since the inverse wavelet transform is removed from the coding loop. However, a major bottleneck for this 
approach is that the classical dyadic wavelet decomposition (named also as the critically-sampled representation) is 

5 only periodically shift- invariant [14][16][17], with a period that corresponds to the subsampling factor of the specific 
decomposition level. Hence, accurate motion estimation is not feasible by using only the critically-sampled pyramid. 
Extensive research efforts have been spent in the recent years to overcome the shift-variance problem of the critically 
sampled wavelet transform. One common alternative is to use near shift-invariant wavelet transforms, and there are 
many solutions in the literature for this type of transforms. However, their main limitation stems from the fact that they 

10 ail imply some degree of redundancy in comparison to the critically-sampled decomposition [1 8][1 9]. An example of a 
video-coding scheme that utilizes such a near shift-invariant transform, namely the complex wavelet transform of Kings- 
bury [18], is presented in [15]. The redundancy factor for this transform is four. Although the coding results obtained 
with this technique seem promising, the main disadvantage is that after performing in-band motion estimation/motion 
compensation (ME/MC), the error frames contain four times more wavelet coefficients than the input-frame samples. 

15 As a consequence, the error-frame coding tends to be inefficient, thus more complex error-frame coding algorithms 
should be envisaged to improve the coding performance. With this respect, it is important to notice that there is a trade- 
off between critical sampling implying efficient error-frame coding and redundancy of the transform implying near shift 
invariance. A completely different solution breaking this trade-off, that is, overcoming the shift-variance problem of the 
DWT while still producing critically sampled error-frames is the low-band shift method (LBS) introduced theoretically 

20 in [16] and used for in-band ME/MC in [14]. Firstly, this algorithm reconstructs spatially each reference frame by per- 
forming the inverse DWT. Subsequently, the LBS method is employed to produce the corresponding overcomplete 
wavelet representation, which is further used to perform in-band ME and MC, since this representation is shift invariant. 
Basically, the overcomplete wavelet decomposition is produced for each reference frame by performing the "classical" 
DWT followed by a unit shift of the low-frequency subband of every level and an additional decomposition of the shifted 

25 subband. Hence, the LBS method effectively retains separately the even and odd polyphase components of the un- 
decimated wavelet decomposition [1 7J. The "classical" DWT (i.e. the critically-sampled transform) can be seen as only 
a subset of this overcomplete pyramid that corresponds to a zero shift of each produced low-frequency subband, or 
conversely to the even-polyphase components of each level's undecimated decomposition. The motion vectors can 
be detected by searching directly in the overcomplete wavelet representation of the reference frame to find the best 

30 match for the subband information present in the critically-sampled transform of the current frame. The motion com- 
pensation for the current frame is then performed directly in its critically-sampled decomposition. Hence, the produced 
error-frames are still critically-sampled. In comparison to image-domain ME/MC, the in-band ME/MC results of [14] 
demonstrate competitive coding performance, especially for high coding-rates. 

[0003] It is an object of the present invention to provide a method and apparatus for performing a subband transform 
35 which is easy to implement than known methods and apparatus. 

[0004] It is also an object of the present invention to provide a method and apparatus for performing a subband 
transform which requires less calculation steps than conventional methods and apparatus. 

[0005] It is still a further object of the present invention to provide a method digital filtering and apparatus for digital 
filtering to generate an overcomplete representation which is easy to implement than known methods and apparatus. 
40 [0006] It is still a further object of the present invention to provide compauter program products for carrying out a 
method for performing a subband transform when executed on a computing device. 

SUMMARY OF THE INVENTION 

45 [0007] The present invention provides methods and apparatus for coding and/or encoding a bit stream comprising 
a representation of a sequence of n-dimensional data structures or matrices, in which n is typically 2. A part of the data 
of one data structure of the sequence can be mappable within predefined similarity critreria to a part of the data of a 
another data structure of the sequence. The similarity criteria may include, for instance, a spatial or temporal shift of 
the data within an image of a video signal such as is used in motion estimation and/or compensation of moving pictures, 

so e.g. video images as well as coders and encoders for coding such a bitstream, e.g. a video signal for use in motion 
compensation and/or estimation. The data structures are typically video frames and a part of the data structure may 
be a block of data within a frame. The shift may be any suitable shift such as a linear translation at any suitable angle, 
a rotation of the data or change of size such as zooming between a part of the data in one data structure and a part 
of the data in another data structure of the sequence. The mapping may be to earlier and/or later data structures in 

55 the sequence. The data structures are typically sequential frames of a video information stream. 

[0008] The present invention provides a method of digital encoding or decoding a digital bit stream, the bit stream 
comprising a representation of a sequence of n-dimensional data structures, the method being of the type which derives 
at least one further subband of an overcomplete representation from a complete subband transform of the data struc- 
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tures, the method comprising: 

providing a set of one or more critically sub-sampled subbands forming a transform of one data structure of the 
sequence; 

applying at least one digital filter to at least a part of the set of critically subsampled subbands of the data structure 
to generate a further set of one or more subbands of a set of subbands of an overcomplete representation of the 
data structure, wherein the digital filtering step includes calculating at least one further subband of the overcomplete 
set of subbands at single rate. In the method a part of the data of one data structure of the sequence can be 
mapped within predefined similarity critreria to a part of the data of a another data structure of the sequence. The 
digital filter may be applied only to members of the set of critically subsampled subbands of the transform of the 
data structure. The method may use a digital filter having at least two non-zero values. The bit stream may be a 
video bit stream. The digital subband transform may be a wavelet. The method may be used in motion compen- 
sation and/or motion estimation of video or other signals which in turn allows compression of the coded video 
signals. The motion estimation may be carried out in the spatial domain or in the subband transform domain. In 
the motion estimation a current frame is compared with a reference frame which may be an earlier or later frame. 
The result of the motion estimation is the selection of one or more subbands from the set of subbands making up 
the overcomplete representation which is or are a best approximation to a shifted version of the reference video 
frame. This or these selected subbands are then used for motion compensation. To make the selection all the 
subbands of the overcomplete representation may be generated, or alternatively, if the motion estimation is known, 
e.g. from the spatial domain, only the relevant subband or subbands need to be generated. 

[0009] The present invention provides a coder for digital subband coding of a bit stream, the bit comprising a repre- 
sentation of a sequence of n-dimensional data structures, the coder being of the type which derives at least one further 
subband of an overcomplete representation from a complete subband transform of the data structures, the coder 
comprising: 

means for providing a set of one or more critically subsampled subbands forming a transform of one data structure 
of the sequence; 

means for applying at least one digital filter to at least a part of the set of critically subsampled subbands of the 
data structure to generate a further set of one or more further subbands of a set of subbands of an overcomplete 
representation of the data structure, wherein the means for applying at least one digital filter includes means for 
calculating at least a further subband of the overcomplete set of subbands at single rate. The coder may be used 
in motion compensation and/or motion estimation of video or other signals which in turn allows compression of 
the coded video signals. A motion estimation module may carry out motion estimation in the spatial domain or in 
the subband transform domain. In the motion estimation module means for comparing a current frame with a 
reference frame is provided; The reference frame may be an earlier or later frame. The motion estimation module 
also comprises means for selection of one or more subbands from the set of subbands making up the overcomplete 
representation which is or are a best approximation to a shifted version of the reference video frame. A motion 
compensation module uses this or these selected subbands for motion compensation. To make the selection 
means for generating ail the subbands of the overcomplete representation may be provided, or alternatively, if the 
motion estimation is known, e.g. from the spatial domain, only means for generating the relevant subband or 
subbands need to be provided. 

[0010] In accordance with the present invention a decoder may receive data structures which are data frames and 
the set of critically subsampled subbands of the transform of the data structure may define a reference frame, and the 
decoder further compries: 

means to map a part of the data of one data structure of the sequence to a part of the data of a another data 
structure of the sequence within predefined similarity critreria and to generate a motion vector for that part and 
means to select a further subband of the overcomplete set of subbands in accordance with the motion vector. The 
decoder may further comprising means to provide a motion compensated representation of the reference frame 
using the selected further subband of the overcomplete set of subbands. 

[0011] The present invention also provides a computer program product comprising executable machine readable 
computer code which executes at least one digital filter for application to at least a part of a set of critically subsampled 
subbands of a data structure to generate a further set of one or more further subbands of a set of subbands of an 
overcomplete representation of the data structure, wherein the application of the at least one digital filter includes 
calculating at least a further subband of the overcomplete set of subbands at single rate. The computer program product 
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may be stored on a data carrier. 

[001 2] ^ The present invention also includes a digital filter module comprising means for application of a digital filter 
to at least a part of a set of critically subsampled subbands of a data structure to generate a further set of one or more 
further subbands of a set of subbands of an overcomplete representation of the data structure, wherein the application 
of the at least one digital filter includes calculating at least a further subband of the overcomplete set of subbands at 
single'rate. 

BRIEF DESCRIPTION OF THE FIGURES 
[0013] 

Fig. 1a is an encoder and Fig. 1b is a decoder with which embodiments of the present invention may be used. 
Fig. 2 shows a representation of a three-level overcomplete DWT decomposition. 

Fig. 3 shows an example of the derivation of the subbands of the overcomplete wavelet transform of level 3 using 
the conventional LL-LBS method. In a scalable coding framework, only subbands ,£7* are available at this level, 
since subbands D Q and D 1 cannot be received by the decoders of this resolution. 

Fig. 4 shows an example $ the derivation of the subbands of the Overcomplete Wavelet Transform of level 3 using 
a prediction-filter method in accordance with an embodiment of the present invention. 

Fig. 5 shows an Overcomplete DWT of k + 1 levels starting from signal X. A "fictitious" pyramid that emerges from 
signal X n is shown in the circled area. 

Fig. 6 shows a memory structure for the concurrent application of the L -tap filters U[n] and W[n] to the input 
sequence {n] using a kernel with L multipliers according to an embodiment of the present invention. The filters 
are linked with the relation = r a U(r^). 

Fig. 7 shows in the upper half the mirroring at the subband edges and the initiation and finalization of the inverse 
transform for level /c-1 that emerges from level k. The lower half shows the mirroring and initiation and finalization 
of the forward transform of level 1 that emerges from the reconstructed signal X. The figure represents the inverse 
and forward transforms with the 9/7 filter-pair, where T H = 9,T G = 7. 

Fig. 8 shows a wavelet video encoder according to an embodiment of the present invention based on 'bottom-up' 
ODWT; motion estimation in spatial domain. 

Fig. 9 shows a wavelet video encoder according to an embodiment of the present invention based on 'bottom-up* 
ODWT; motion estimation in wavelet domain. 

Fig. 10 shows a wavelet video decoder in accordance with an embodiment of the present invention based on 
'bottom-up' ODWT. 

Fig. 11 shows a wavelet video encoder in accordance with an embodiment of the present invention based on 
'bottom-up 1 MC; motion estimation in wavelet domain. 

Fig. 1 2 wavelet video decoder in accordance with an embodiment of the present invention based on 'bottom-up' MC . 
Fig. 13 shows three wavelet decompositions for different translations of the input signals X(z). 
Fig. 1 4 shows frame 5 of a football sequenve decomprtessed at 760 kps in different resolutions. From top to bottom: 
original size, half resolution and quarter resolution. 

Fig. 15 shows a schematic representation of a telecommunications system to which the present invention may be 
applied. 

Fig. 16 shows a subband coding circuit in accordance with an embodiment of the present invention. 

Fig. 17 shows a further subband coding circuit in accordance with an embodiment of the present invention. 

DEFINITIONS 

[0014] 

DWT: Discrete Wavelet Transform 
SBC: Subband Coder 
SBD: Subband Decoder 
EC: Entropy Coder 

CODWT: Complete-to-Overcomplete DWT 

ME: Motion Estimation 

MC: Motion Compensation 

MVs: Motion Vectors 

MVC: Motion-Vector Coder 

I DWT: Inverse DWT 
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ED: Entropy Decoder 

MYD: Motion-Vector Decoder 

Single rate: a calculating a subband without upsampling or downsampling. 

Level: refers to a level of a subband pyramid containing the subbands of the subband transform of a data structure 
such as an image 

Level-by-level encoding: in a multiresolutional, multilevel coding scheme, encoding each level of the subband 
transformed data structure to allow transmission of that level (resolution) independently of other levels (resolutions). 
Level-by-level decoding: in a multiresolutional, multilevel coding scheme, decoding each level of the received bit 
stream to allow display of that level (resolution) independently of other levels (resolutions). 
Scalability: the ability to decode a coded bitstream to different resolutions 

Temporal scalability: ability to change the frame rate to number of frames ratio in a bitstream of framed digital data 
Quality scalability: the ability to change to the quality of a display 
Overcomplete representation 

Critically-sampled representation: a transform having the same number of coefficients as the data structure being 

transformed 

LBS: Low-band shift 

Baseline quality-layer: minimum information to be transmitted to provide a reconstruction of a data structure in the 
receiver. 

DETAILED DESCRIPTION OF THE ILLUSTRATIVE EMBODIMENTS 

[0015] The present invention will be described with reference to certain embodiments and drawings but is not limited 
thereto but only by the attached claims. In the following reference is made to subband transforms. Included within 
these techniques are: wavelet, Discrete Fourier transform, Fast Fourier transform. In the following the present invention 
will mainly be described with reference to in band motion compensation and estimation of framed data but the present 
invention may find wider application than this application alone. Further, in the following the invention will be illustrated 
with respect to linear translations between parts of video frames, but the present invention is not limited to linear 
translation models for motion compensation and estimation. 

[0016] During encoding of video digital signals an original video image information stream is transformed into another 
digital representation. This digital representation may be stored on a suitable data earner or transmitted to a remote 
location. The aim of coding is to provide an advantage, e.g. data compression and/or scalability of the data stream. 
The aim of decoding such an encoded signal is to reconstruct the original video information as economically and as 
well as possible, e.g. a lossless or, if it is preferred, a lossy reconstruction. The arrays of pixels of such close frames 
often contain the same luminance and chrominance information except that the coordinates of pixel positions in the 
arrays are shifted or displaced. Shifting in position within the array is a function of time and defines a motion of these 
pixels within the arrays. The motion may be approximated by a motion vector. The encoding is based on the fact that 
at least parts of temporally close video frames either in the forward or backward direction are often quite similar except 
for motion. This similarity means that repeat transmission of this moving data in each frame is not required, it is only 
necessary to transmit a code of a shift of the data from a previous or subsequent frame, e.g. a motion vector. 
[0017] Typically, the motion used for motion estimation and compensation is a linear translation, however more com- 
plex motions or changes in pixel distributions can be considered as a basis for motion compensation and estimation. 
Hence, the present invention includes within its scope alternative motion estimation and compensation modeis, e.g. 
linear translations, rotational translation of pixels, zooming of one part of an image compared to another. Hence, the 
model used may be generalized to say that there is a mapping between a part of one video frame and another video 
frame, this mapping being determined by predefined similarity criteria, that is the criteria which are to be used to 
determine that there is similarity between the parts of the two frames. This similarity may include linear translation, 
rotational translation, zooming between frames. 

[0018] In one aspect the present invention provides an approach for in-band ME/MC wherein prediction-filters are 
used. The algorithm can achieve mathematically the same result as the LBS algorithm of [14]. The LBS algorithm is 
suitable for coding and/or encoding a bit stream comprising a representation of a sequence of n-dimensional data 
structures, in which n is typically 2 or 3, and where a part of the data of one data structure of the sequence maps within 
predefined similarity critreria to a part of the data of a another data structure of the sequence and is of the type which 
derives at least one further subband of an overcomplete representation from a complete subband transform of the data 
structures. The use in accordance with embodiments of the present invention of prediction-filters for in-band ME/MC 
overcomes any shift-variance problem of the subband transform such as DWT. Critically sampled error-frames are 
produced. With this algorithm, the overcomplete wavelet representation of each reference frame is produced directly 
from its critically sampled wavelet representation by using a set of prediction filters. In this way, there is no need to 
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reconstruct the spatial domain representation of each reference frame, and to produce from it the corresponding over- 
complete representation, as is the case of the LBS method. When used in a level-by-level manner (i.e. ensuring spatial 
scalability), this is an advantage over the LBS method. Level-by-level processing provides scalability as each level 
may be transmitted and/or received and processed independently. Fast level-by-level calculation of the overcomplete 
discrete wavelet transform starting from the critically sampled pyramid is disclosed including a generalized form for 
the prediction filters for an arbitrary number of decomposition levels. Some symmetry properties between the prediction 
filters are disclosed, specifically, for biorthogonal point-symmetric filter-pairs that are the most-performing types of 
wavelets in still-image coding, leading to efficient implementation. A comparison from the computational point of view 
of the invention with the LBS method is given. 

[0019] An example of a communication system 210 which can be used with the present invention is shown in Fig. 
15. It comprises a source 200 of information, e.g. a source of video signals such as a video camera or retrieval from 
a memory. The signals are encoded in an encoder 202 resulting in a bit stream, e.g. a serial bit stream which is trans- 
mitted through a channel 204, e.g. a cable network, a wireless network, an air interface, a public telephone network, 
a microwave link, a satellite link. The encoder 202 forms part of a transmitter or transceiver if both transmit and receive 
functions are provided. The received bit stream is then decoded in a decoder 206 which is part of a receiver or trans- 
ceiver. The decoding of the signal may provide at least one of spatial scalablity, e.g. different resolutions of a video 
image are supplied to different end user equipments 207 — 209 such as video displays; temporal scalability, e.g. 
decoded signals with different frame rate/frame number ratios are supplied to different user equipments; and quality 
scalability, e.g. decoded signals with different signal to noise ratios are supplied to different user equipments. 
[0020] A potential' architecture of a wavelet-based video codec performing in-band motion estimation/motion com- 
pensation (ME/MC) with which a first embodiment of the present invention may be used is given in Fig. 1 whereby Fig. 
1 a shows the encoder and Fig. 1 b the decoder. This architecture can be seen as an extension of the "classical" archi- 
tecture of transform-based video coders/decoders. In this scheme there are two modes of operation as determined by 
the setting of the switches 16, 18, as explained in the following. In the intra-frame coding mode, the current input frame 
4 is first of all wavelet transformed using a suitable compression technique such as the DWT module 2 of Fig. 1 , the 
resulting wavelet subbands are quantized and encoded using an embedded intra-band coding technique, (SBC module 
8) and the result is entropy-coded using the EC module 1 0. The embedded intra-band coding and the entropy coding 
are performed in a level-by-level fashion to guarantee resolution scalability. Furthermore, this process is performed 
until the target bit-rate for the currently coded frame is met. The second coding mode is the inter-frame coding mode, 
where each reference frame is first of all wavelet transformed using the DWT module 2 of Fig. 1 . As in the previous 
mode, the subbands of every decomposition level are encoded using the intra-band embedded coding technique in 
SBC module 8 and entropy-coded in EC module 1 0. The SBD module 1 2 performs the decoding operation and produces 
the base quality layer of the reference frame, which is stored in a memory buffer 20. The complete-to-overcomplete 
DWT module 22 (CODWT) produces, for each level, the set of one or more subbands of the overcomplete represen- 
tation of the reference frame starting from the subbands of the critically sampled pyramid. These are stored in a buffer 
memory 24. The resulting n subbands per level are then used during the motion estimation which is performed by the 
block-based ME module 30 to find the best match between the subband-information present in the current frame, 
received at module 30 from the output of DWT module 2, and the overcomplete representation of the reference frame 
received from the buffer 24. A block 26 is selected using selecting means in the motion compensation module 28 or 
the motion estimation module 30 from the buffered subbands of the reference frame that represents the best match is 
used for the motion-compensation propess which is performed by the MC module 28. The resulting motion-vectors of 
every decomposition level are sent to the motion -vector coder (MVC module 32). The output of the motion compensation 
is an error frame in the wavelet domain which is generated by subtracting the output of MC module 28 from the subband 
transformed current frame in subtracters. This error frame is intra-band embedded coded in SBC module 8 and entropy- 
coded in EC module 10 in a level -by- level fashion. To generate the reference frame 20 from the subband decoded 
intra-band coded signal the output of SBD 12 is added to the output from MC 28 in adder 14. This reference frame is 
used in the next iteration. The dashed line shows a less preferred embodiment in which the error frame is supplied 
directly to the adder 14. 

[0021] As shown in Fig. 1b, the decoder operates exactly in the mirror fashion, the intra-frame or inter-frame decoding 
being determined by the switch 52. Hence, in the intra-frame mode, the resolution-scalable coded frame is received 
and the bitstream parsing can cease at any resolution or quality level (provided that the minimal base-quality layer is 
received). The next operations are entropy decoding in ED module 32 followed by subband decoding in SBD module 
34 and inverse DWT in IDWT module 40. At this stage, the frame corresponding to the specific operational settings e. 
g. quality and resolution, is reconstructed. 

[0022] In the inter-frame mode, the error-frame is received at the desired quality level at the ED module 32 and the 
corresponding motion vectors of each decomposition level are received and decoded as well by a MVD module 50. 
Similar to the encoder, the complete-to-overcomplete DWT of the reference frame 46 is constructed in a level-by-level 
manner In CODWT module 48. The key difference is that since the motion vectors are known, only Individual blocks 



EP 1 294 196 A2 



44 needed for the motion compensation which is to be performed in module 42, are calculated by the CODWT module 
48. In the motion-compensation phase carried out in module 42, these blocks 44 are used to reconstruct the predicted 
frame. The predicted frame is added to the error-frame 36 of the current decomposition level in adder 38. To ensure 
drift-free quality scalability, the CODWT unit 48 operates on the base-quality layer for every decomposition level. Hence 
5 the decoder reconstructs the identical overcomplete representation per level as the encoder. In this manner, the decoder 
is free to stop decoding at any resolution or quality level. 

[0023] From the description of the previous subsection, it can be seen that the CODWT module 22, 48 is a key part 
of the ME/MC subsystem of the scalable video-codec. 

[0024] Fig. 2 shows an example of the 1 -D overcomplete pyramid for three decomposition levels as constructed by 
w the LBS method. Initially, the input signal X is decomposed into two sets by retaining separately the even or odd 
polyphase components of the undecimated decomposition, or equivalent^ performing two "classical" wavelet decom- 
positions; one in the zero-shifted input signal and one in the unit-shifted input. Each of the low-frequency subbands of 
the two decompositions is further analyzed into two sets by performing again an undecimated wavelet decomposition 
and retaining separately the even and odd polyphase components and so on. The 2-D overcomplete pyramid is con- 
15 structed in the identical manner, with the application of the LBS method in the input-subband rows and the columns of 
the results. Hence, to facilitate the explanation, the description in the following is restricted to the one-dimensional 
case for clarity's sake, with the extension in two dimensions following the row-column approach. By definition S 0 , S 1 

denote the even and odd polyphase components of signal S respectively (named also type 0 and type 1). Each low- 

20 frequency (average) subband of decomposition level / is denoted as A 1 , where x is a binary representation that shows 

the subband type, with the Most-Significant Bit (MSB) denoting the M Mother"-subband type and the Least-Significant 

Bit (LSB) denoting the current-subband type. For example, subband A ^ denotes the low-frequency subband of level 

3 that has been produced by retaining the even polyphase components of decomposition level 1 and the odd polyphase 

25 components of levels 2 and 3. Similar notations apply for the high-frequency (detail) subbands D 1 . Note how the LSB 

of the binary representation denotes a "turn" in the pyramid (0 for "left", 1 for "right") in comparison to the previous- 
level subband ("parent"). Thus, for every decomposition level, the subscript bits are shifted to the left and a new LSB 
is entered, depending on the polyphase grid that is retained from the decomposition (even, odd), in a scalable-coding 
3Q framework, the key-difference is that the subband-transmission and decoding occur in a bottom-up manner, every 

decoder receives the coarsest- resolution subbands first (i.e. subbands A 2 ,0?) and is free to stop the bitstream-parsing 

at any time after the baseline quality-layer has been received for each level. In this way, the quality and resolution of 
the decoded video can vary accordingly. Under such a progressive decoding environment, the LBS method is modified 
to perform a level-by-level construction of the overcomplete representation (denoted by LL-LBS), starting from the 
subbands of the critically-sampled transform of each coded level. Such a situation is illustrated in Fig. 3 for the LL-LBS 
method applied on a 3-level decomposition; firstly, three inverse wavelet transforms are performed starting from the 

subbands A 2 , & t as shown in the left part of Fig. 3. Notice that in this situation, the subbands & and D 1 are not present 
oo oo 

(i.e. are not received at the decoder site), and as a consequence, the subbands shown in Fig. 3 are not identical with 
the corresponding ones of Fig. 2. Subsequently, from the highest-resolution signal X, all the subbands A* t C?jG[\ ,7], 
/ G Z are constructed by performing the forward transforms shown in the rest of Fig. 3. It must be noted that the low- 
frequency subbands A* are constructed only if Zeis the coarsest-resolution level of the decomposition. In all the other 

cases, only the high-frequency subbands 0^, with /G[1,2*-1], /GZ are needed. This is imposed by the fact that no 

motion estimation or compensation is performed for the subband A* if Zeis not the coarsest decomposition level. Hence, 
in the example if after the construction of the overcomplete transform of level 3, one more resolution level is received 
so (i.e. subband D^), then the LL-LBS operates again in the same fashion, but constructs only the subbands D^, with / G 

[1 ,3], / G Z Although approximations of the subbands can already be calculated during the calculations of level 3, 

at this time the subband E? is not available; as a consequence, these approximations are obtained based only on the 

55 subband A* and hence they do not have the best accuracy possible. This observation indicates a significant difference 

between the LL-LBS and the LBS methods: because of the bottom-up level -by-level construction, the high-frequency 
subbands of the higher-resolution levels (levels 2 and 1 in the example) are not available when the current level is 
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processed (level 3). Hence, the resulting overcomplete representation of each level obtained with the LL-LBS method 
is not identical to the one constructed with the LBS algorithm. Actually, in its original form, the LBS algorithm creates 
the overcomplete representation under the assumption that all the subbands of the critically sampled pyramid are 
available, and this is not always the case in a resolution-scalable framework. This means that the LBS method requires 
s the receipt of all resolutions in order to create the overcomplete representation. This places a severe limitation on the 
time required to decode an image - ail resolutions must be available. However, once ail the information has been made 
available at the decoding time, the level-by-level LBS produces the subbands of the overcomplete representation with 
the best accuracy possible while simultaneously ensuring drift-free, full resolution-progressive decoding (i.e. spatial 
scalability). 

w [0025] In accordance with embodiments of the present invention, it is not necessary to receive the highest resolution 
information in order to generate a useful overcomplete representation. In accordance with an aspect of the present 
invention, the overcomplete representation can be obtained by the application of prediction filters. An example of the 
derivation of the subbands of level 3 with the prediction-filters method is given in Fig. 4. It can be noticed that the 

overcomplete representation is "predicted" in a level-by-level manner using the sets of filters F^ Rl with OG[1 ,3],QGZ 

and flE[0,15],fleZ indicated under the subband-pairs of Fig. 4. However, no upsampling or downsampling is performed 
with this algorithm and this leads to substantial complexity reductions. Where no upsampling or downsampling is used, 

the calculation is said to be at*single rate. The form and utilization of filters f~ is explained in the following, where, in 

20 accordance with an" embodiment of the present invention, the general proof for an arbitrary-level complete-to-over- 
complete transform is presented. In the same manner as in the LL-LBS method, for the higher-resolutions (decompo- 
sition levels 2 and 1 in the example of Fig. 3), only the high-frequency subbands are constructed, it must be noted also 
that if applied in a level-by-level manner, both methods produce identical results. In the general case of an overcomplete 
representation with levels shown in Fig. 5, the binary representation of the subband indices becomes impractical, 

25 and thus they are simply denoted with their decimal equivalent, in the form Z bp, with b f - {0, 1 }. In addition, as shown 

in Fig. 2, the overcomplete pyramid is separated into the "left-halt and "right-half* pyramids respectively. These parts 
correspond to the two "Mother" subbands containing the even and odd-polyphase components respectively of the 
undecimated decomposition of the original signal. To conclude the notations, in the Z-plane expressions, frequently H 

30 (z), >4^(z), F^z) are simply denoted as H } A^, F^ respectively, to reduce the expressions' length, while (H)o»(^)i denote 
the even or odd polyphase components (similar applies for H f ^H iA for filter H,). 

[0026] in this section a generic framework is presented for an embodiment of a prediction filter. This framework 
allows the complete to overcomplete transform derivation for an arbitrary decomposition level. The mathematical der- 
ivations are performed for the 1-D overcomplete representation; nevertheless, as it will be shown later, the extension 
to 2-D is straightforward following the row-column approach or equivalent thereto. Firstly, the subbands of decompo- 
sition level ko\ the overcomplete representation are expressed as a function of the critically-sampled wavelet decom- 
position (i.e. the subbands A k t D l , with /£ [1,/cj, IG Z). From this generic mathematical formulation, the level-by-level 
overcomplete transform of levePk is readily extracted as a special case, in which the overcomplete subbands are 
calculated using only the critically-sampled representation of the same level. Finally, the symmetry properties of the 
prediction filters for every decomposition level are given, which allow their efficient implementation. 

Derivation of Subbands of Decomposition Level k from the Critically-sampled Pyramid ot Level k - the Prediction Filters 
in accordance with an embodiment of the present invention 

45 

[0027] The proof of the general form of the invented prediction filters is as follows in which the prediction filters for 
the decomposition levels E = 1 ,2,3 are derived. 
The proposition P (1) corresponding to E= 1 is: 
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P(l): 



(1) 
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For level E = 2, the set of prediction propositions P (2) is: 
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\d?=f1a;+f1d 2 0 

\a\ = fZ4 + f?dI+{h ^D\\ 

\Dl=Ft/$+FlDl+{G*FlD\\' 
At^F^+FtDl+iH.FlDW 
D] =F£A^ + F 7 2 D% -h(CF l l D ] Q ) l 



[0028] The generalization and the proof for an arbitrary level E = ft will be attempted with mathematical induction. 
Thus we assume the set of prediction propositions P (ft) for an arbitrary level ft to be true and based on them we derive 
the propositions P (k+ 1) and the filters for level k + 1 . The formulas derived for level ft + 1 are true if, and only if, they 
can be derived from level ft by replacing k with ft + 1 . In addition, if they are true for level ft 1 , then according to the 
induction principle, they are true for any level £ 

[0029] Therefore, let us assume that the propositions for the levels E = 1 ,2, ... k - 1 , ft with ft £ 2 are ail true. The 
proposed set of propositions P (ft) corresponding to the level E = ft of the overcomplete decomposition pyramid is: 

(3) 



A pictorial representation of the subbands of level ft and the corresponding prediction filters is given in Fig. 5. In these 
equations, x E [1 ,2* - 1], x S Z denotes the subband index at level ft, and it is written as: 



x=2 / + p, (4) 



where /is given by: 



/ = i_ log 2 xJ, (5) 



and L aJ denotes the integer part of a. From (4) and (5) if follows that / E [O.ft -1], / E Z and p E [0, 2'-1 ], p E Z for any 
subband index xtaken in the interval [1, 2*- 1]. Considering 1 in the set of equations (3), b h 0 £ /</- 1 are defined 
as the binary values from the binary representation of p, given by 



, with fc/E {0,1}, while i(m) are filter indices defined as 



i(m) = 4 



2" 



+ 1 



, for m E [1 m E Z. In the particular case of /= 0 corresponding to k = 1 and x= 1 we set b f = 0, Vi EZ and F B = 0 
for any indices A < 0 or B £ 0, to ensure that the set of equations (3) is identical with P (1 ) given in (1). A 
[0030] The proposed set of propositions P (ft) can be divided into two sets of equations P L (ft) and P R (ft) with P (ft) 
= p l (*)^ p r (A), each set corresponding to the subbands of the "left-hair and "right-half parts respectively of the 
pyramid of level k (see Fig. 3). Proving by induction the set of propositions P (ft) is equivalent to proving by induction 
each set of propositions P L (ft) and P R (ft). 

[0031] The set of propositions P L (ft) corresponding to the left side of the pyramid of level ft, with ft 5 2 is obtained 
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from (3) for x G [1 ,2* 1 -1]: 



P L (*) : 



(6) 



Note that for k = 1 there are no predictions to be made for the left side of the pyramid, that is P L (1 ) = 0. The remaining 
iQ set of propositions P R (*) corresponding to the right side of the pyramid of level Ac is obtained for x G [2*' 1 ,2*- 1], Notice 
that from (5) it results that /= k - 1 therefore equation (4) is equivalent to: 



x=2*" 1 +p. (7) 
Since xG[2 k -\ 2 k - 1], it results from (7) that p e [0, 2*" 1 - 1], pe Z. Replacing l=k- 1 in (3) yields: 



(8) 

We follow in these equations the same definitions as given in equation (3), that is, b b O^k-2 with kz 2 are the binary 
25 values (£?/ G {0,1}) from the binary representation of p given by 

»-2 



, and /(m) are filter indices given by 



i(m) = 4 



m€[1,/f-1]. In the particular case of k= 1, we set fy=0,V/GZ and = 0 for any indices 4 < 0 or 0£ 0, to ensure that 
P R (1) given in (8) is identical with P (1) given in (1). 

[0032] For the particular case of biorthogonal point-symmetric filter-pairs, the filters of decomposition level k 
needed to calculate the subbands of the "right" side of the pyramid of level k, with /= 0,1 ,..,2* 2 - 1 , are: 



ft,-f**lj>-**lirtl,y ( 9 ) 



• ^Irt-'^A,,,. (11) 
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< 15 > 

^7- rt \l + ^ rt V (16) 



For completeness, we give the filters of the proposition P (1). They have the form: 



Fq = Dei 1 (H, G 1 - zH 0 G 0 ), f\ = Def 1 (zH 0 H 0 - H, H 1 ), (1 7) 



= Def 1 (G 1 G 1 -zG 0 G 0 ), fJ = Def\zH 0 G 0 - (18) 



' 5 where Def is the determinant of the analysis polyphase matrix hL (z), given by: 



zj M -f^W ">M1 rim 



[G 0 (z) G t (z)y 

Let us prove now that the propositions P L (k + 1 ) formulated for level E = k + 1 are true. Notice from Fig. 5 that the 
"left" side of the pyramid with k + 1 decomposition levels (original pyramid) emerges only from the subband A Q . Hence, 

25 is is 

one can define a new, "fictitious" pyramid with subbands A fjJ ,D njs f G[1 ,/c], j s e[1,2'M] that emerges from an input 

signal X n with X n - a\ This pyramid is shown in Fig. Sin the dashed line-enclosed area. One can notice that the entire 

decomposition level / $ of this "fictitious" pyramid is equivalent to the "left" side of level i s +1 of the original pyramid 
emerging from X, with the relationship: 

A If 1 =4j s « Ojf = D^, i s G [1 , *], j s G [1 ,2* -1] . (20) 

Notice that / s 6 [1 ,Aj, therefore the set of propositions P (k) can be applied for the level k of the "fictitious" pyramid, 
since they are true by assumption. Hence, by using equation (3) one can write: 



45 Dk fl , = 0*i» + *M* + < G • 0>a<! \ + • >* k + - + (22) 

in which all indices have the same definitions as in equation (3). 

[0033] By performing the transformations of variables specified by equation (20), equations (21) - (22) become: 

50 ~* 



A**x = /=4p A^ A + F4p+10*O 1 + (W-^jDJ)^ + 



55 



[H^u^O^]^^ (23) 
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[G(H^i { 2)D^) b ) b +.AG(HUH ^ h Dr A 'o) b ) b ..) b ,) bQ (24) 

10 1 ' M '-2 

5 

Notice that these equations are identical to the set of equations (6) in which Zeis replaced with k + 1. This means that 
• the propositions P L (k+ 1) are true. Additionally, only filters of levels 1 ,2,.., kare used, which are true by the assumption 
of level k. 

1Q The propositions P R (k + 1) for the "right" side of the pyramid remain to be proven. 
In addition the form of the filters has to be proven as well. 

[0034] The set of propositions Pr (/c+1) are formulated by replacing frwith k + 1 in equation (8): 
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(25) 

To prove these propositions, let us start by performing an inverse wavelet transform in order to calculate the A subband 

in function of the/** +1 and subbands: 

o o 

At ^z[H(z)Ai^(z l )^6(z)D^(z 2 )} . (26) 

Since both A k and D* 0 subbands are known now, we can apply any of the propositions of level k (involving the F 
filters), since ?hey are true by assumption. Hence, we can calculate any subband A k , with x 6 [2 k ^ t 2 k - 1] by using the 
set of propositions P R (k) given in equation (8). By replacing (26) in (8) we obtain: 

A k x (z) = zF*,(z)H(z)Ai* l (z 2 ) + zF A k p (z)G(z)D^(z 2 ) + F A k ^(z)D k (z) + T{k , (27) 



with: 



nk-i^H-tfmlfQh +[H(H.F /r / 2 (2)D*o 2 ) 6 } b +..+[H{HUH.f] {k Cfo b ) b „) b ] b , (28) 

0 10 ' " *-2 *-3 1 0 



and /(m) defined as for equation (8). The "tail" 7{/r-1 ,z) denotes the contributions of levels 1 ,2,.., . 

[0035] In order to calculate the A ,Cr subbands (even-numbered subbands of level k + 1 ), we need to perform 

a single-level forward transform, ref&fning'fhe even samples ("classic" transform): 



LA 4 ;' J ~ 2 [ g 0 ( z )h- 2^(7,(2) G 0 (z)-*-*G,(z) . 



A* (-**>. 



(29) 



To derive the terms in (29) we need to evaluate AJ^z*) and d By replacing z with & in (27) we get: 

4 = zXf: p {zX)h(zX)4« (z) + ^(^^yflf *'(x) + + r(* - U*) 

(30) 

The filter F (z % ) can be written using the Type I polyphase representation as: 

4p 
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(31) 



whilei(2 )4 ) ande (2*) can be substituted by the Type II polyphase representation, as follows: 



#(z*) = #,(z) + 



(32) 



1Q, 



G(z^) = G,(z) + z"*G 0 (z) , 



(33) 



Equations (32) and (33) are equivalent to: 



15 



H(2 y >) = Der\-G a {z) + z-*G x (z)} , 



(34) 



20 



G(z y >) = Der'[l/,(z)- J*«i(x)], 



(35) 



due to the relations: 



25 



tf 0 (z) = 0er' G,(z), tf,(z) = -Def' <7 0 (z), 
G 0 (z) = -Der"' • //,(z), <5,(z) = Der' • // 0 (z), 



(36) 



30 



between the polyphase components of the decomposition and reconstruction filters in a filter-bank. These relationships 
can be immediately verified by validating the perfect reconstruction condition. In the following, we always assume that 
the filters H and G are properly shifted so that Def 1 = -1 , in order to simplify the expressions, shows such an example 
for a biorthogonal point-symmetric filter-pair, namely the 9/7 transform of Cohen, Daubechies, Feauveau. 
[0036] By replacing equations (31), (34), (35) in equation (30) we obtain: 



35 



A*(S)=AFt po +Z*F$ p ,W^^ (37) 



where 



40 



T{k,z*} = /< p+1 (^)DS(zV 7K*- 1,A 



(38) 
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is the "tail" that includes the contribution of level k. k 

By grouping together the factors of the polyphase components of F in equation (37) we obtain: 



A k x (z*) = [(z% - Gi)f* 4po + (G 0 - z^G^ JA^o + 
[(H, - z*H 0 )>Vo + (z*Hi - H 0 )fJ p1 JD*S 1 + T(k,z*) 



We denote the factors of the two subbands A k ^ ,D** 1 of equation (39) as: 



(39) 



(40) 



(41) 
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By using these notations, equation (39) is equivalent to: 



A k JL?) = L+^A*o + L + ? +1 otf + T(*,z\ (42) 



5 

Similarly as above, we can calculate A k {-&) by replacing z with (-z*) in equation (27): 
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(43) 

The filter F* (-2*) can be written using the Type I polyphase representation as: 

^(.^=fJ po (^M P(1 w. <"> 

while for point-symmetric, bioijhogonal filter-pairs, i(-z % ) and d can be substituted by the Type II polyphase rep- 
resentation as: 

> 

#(-«*) -A,(s)-s-*A.(s), (45) 



d(-x* ) = G, (z) - z-*C 0 (z) , (46) 



equivalent to: 



H (-z*) = Z?-'[-G 0 (z) - z'*G, (z)] , (47) 

G(-2^)«D-[/f ( ,(z)+z"^,(r)], (48) 

due to the relations mentioned before - equation (36) - for the polyphase components of the decomposition and recon- 
struction filters in the filter-bank. 

By replacing equations (44), (47) and (48) in equation (43) with Oef 1 = -1 we get: 

■ 

^ ( -/)=.^ p , 0 ^^ (49) 

where Uk r &)=F* 4 ^(-&)D k 0 (-&)+T( i k-'\r&) Is the'tair that includes the contribution of level k. By grouping together 
the factors of the polyphase components of /=* 4p in equation (49) we get: 

A%-J*) = +(G 0 + / H 0 1 )fJ Pil M l S 1 + 

We denote the factors of the two subbands as: 

o o 

L'm = (*^<3 0 - G,)^ + (G 0 + /"O,)^,,. (5D 

L ° +1 - (H, + ^H 0 )^p.o + (•* *«i * H o)^ P .r ( 52 ) 
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By using these notations, equation (50) is equivalent to: 



(53) 



The final expressions of A M .D** 1 are obtained by replacing equations (42) and (53) in equation (29), as shown in 
the equation below: 2 * 2x 



10 . 



r^;'i i\h 0 (z)+z-*h,(z) // 0 ( 2 )-^//,( 2 )ir L^'+c.or +?-(*.**) i 



(5 Equation (54) shows that the calculation of the subbands -4* +1 , f/ + J consists of separate calculations of factors like 
U{(H 0 +z*H,)L+ A HHo-r^)L A ] and Vi(H^z*H^L* D +{H 0 -z^H,)L ° ], multiplying the subbands A™ and D** 1 

Jf+1 /f+1 Jf+1 ^+1 

respectively. These factors correspond actually to the prediction filters of the even-numbered subbands of level . 
The calculation of these factors is done in the following. 

20 

a -A Jf+1 

Part 0: Replacement of L and L in equation (54) for the calculation of A 
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2F*? p - (H 0 + z*Hi)[(z*G 0 - G,)F* pQ + (G 0 - z*Gi)/=J p ,}+(H 0 - z'*H,)[-z*G 0 - G^ 4pfi + 

+ (G 0 + z 11 ©,)^,] => rt p = ^p, 0 • ^Wy- < 55 > 



Part f; Replacement of /. +D and L *° /n equation (54) for the calculation of A M : 

fr+1 fr+1 2x 



2/^ 1 p+1 = (H 0 + /"h^H, " ^«o)^ P ,o + - W 0 )^,,]+(Ho - z' H "i)K"l + ^«o)^p fl + 



Part 2: Replacement of L** andL' A in equation (54) for the calculation of : 

*+/ 2x 



^p* = ( G o + ^' H Gi)K^ G o - <3i)^p.o + ( fl o " /*et)^ p>1 l + (G 0 - z*G,M-z*G 0 - G,)^ + 



Part 3; Replacement ofL*° and i' 0 in equation (54) for the calculation of £/ +1 : 

/f+1 /f+1 2x 



2f*S 1 p+3 = (G 0 + z' H Gi)K«i " ^Ho)^ P , 0 + (*'* H 1 ' "oJ^WQj • *'*G,)K«1 + ^"o^o + 

- H 0 )^p.,l => ^ 1 p+3 - 'Vo = < 58) 

1 

50 By definition p = 0,1,.. ,2 fr1 -1. 

We can complete now equation (54), and hence the propositions for the even-numbered subbands of the "right" 

side of the pyramid, at decomposition level : 
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and 



with: 



W.*) = Fj p+1 D* + (H.tf m rf) bo * l*H.tf n d$) bt \„ Q + MHM.\H.F\ k , ) Dl) bk J bk ^ b ) bo . (61) 
Define y = 2x. Since x e [2* 1 ,2 k - 1) it follows that y e [2*,2* +1 - 2]. The equivalent of equation (7) is: 



y = 2%<7. (62) 
with q E [0,2* - 2]. Also, the definition of y and equations (7) and (62) imply that: 

q = 2p. (63) 
This equation shows that o; is an even number, therefore its binary representation is given by 



0 2<eO C « *^ 



, with c f E {0,1}, V/ E [1 ,/c- 1] and Cq =0. The binary representation of p is 



, therefore by using (63) one can write: 



From (64) it follows that c, = fc M for any /E[1 ,/f-1]. 

By using these observations and by replacing the expression of the "tail" T given in equation (61), one can write 
equations (59) and (60) as follows: 

*{H(HUH • F l m D l 0 ) e „ ) tfw U . (65) 



and 
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- O**' + + t G -' F 4 j 1 L 1 £) o% + W • FyW), ]^ + [G(//(ff • F^Z)*-')^ ]^ +.. + 



(66) 



10 



where y(m) are defined as 



70») 



-4*J 



+ 1 



/5 



20 



25 



,for any me[1,/c]. 

Equations (65), (66) are equivalent to equations (25) for the even values of x (even-numbered subbands). Hence, 
in this case the propositions P R (k+ 1) are true. 

The proof of P R (k + 1 ) for odd values of x is done in a similar manner In order to calculate the subbands A * +1 , 
D k * , we perform a single-level forward transform retaining the odd samples ("complementary" transform)?** 1 



2x+1 





^ 1 




°2 



tf,(z) + Z%,(z) //,(z)-Z*// 0 (z) 

G,(*) + **G,(z) G,(z)-z*G 0 (z) 



4(z*) 

.4<-z*>. 



(67) 



Notice that the terms A (z*),A are already calculated in equations (42) and (53). Replacing their expressions 
in equation (67) yields: 



30 



^1 



I 

2 



7/ 0 (z)+«%,(*) //,(z)-z% 0 (z) 
G () (z) + z% l (z) G,(z)-z*G 0 (z)_ 



(68) 
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Equation (68) shows that the calculation of the subbands A , D consists of separate calculations of factors 

2x+1 2x+1 



like !4[(tf 0 + ^^)L* A + (*1 - &H$L'* \ and #[(H 0 + + (H^ - ^H 0 )Z/^] f multiplying the subbands 

A M and 0* +1 respectively. These factors correspond actually to the prediction filters of the odd-numbered sub- 
oo 

bands of level k+ 1 . The calculation of these factors is done in the following. 

+A 'A X+1 

Part 4: Replacement of L and L in equation (68) for the calculation of A 

K+1 2x+1 

zr ** M = < H 1 + ^H 0 )[(^G 0 - GJF% pfi + (G 0 - Z*Gi)F* p ,] + (H, - ^H 0 )[{-Sg 0 - G^)F^ pQ + 

+ (G 0 + 1*6,)^,] => /^ 1 p+4 = F* p1 + Fjf* po . (69) 

Pa/t 5: Replacement ofL +D and L '° in equation (68) /br fhe calculation of A : 

/f+1 2x+1 

+(-**«, ' - H 0 ) F* p f1 ] =» F*8 1 p+5 = Fj F* p „. (70) 
Part 5: Replacement of L and L in equation (68) for the calculation of D 

*+1 2x+1 



•A 



40 
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2 ^V6=( G i + ^ G o)K^Go-Gi)^ P .o + ( G o-^G 1 )/^ p J+(G r z^G 0 )^2 H G 0 -G 1 )/< po+ 

HG 0 *z*G % )ft„\~f*B „ = F^ R0 . (71) 
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Part 7: Replacement of L* D andL"° in equation (68) for the calculation of 2x+v 

+(-z % H y Hi)F* pA \ =» f*S 1 p+7 =f* p1 + ( 72 ) 

By definition p= 0,1 ,..,2*- 1 -1. 

We can complete now equation (68), and hence the second proposition of level k+\ : 

All = /C 4" + O**' + + ** ». )^(*, ^ ) + - *%o)*X*. )] => 



and 



=C«4*' + ±K G > +x x o.)i , (*.^)+W -* K c 0 )7-(A,-z*)] 



(74) 



with 



7KM= ^ p+1 d5 + («- P*"/(l)D*o 1 ) b +[H(H-/w 2 (2)D*o 2 ) 6 J b +..+[H(H(..(H • F? dJ)„ )„..)„ ] 6 • (75) 

Define y = 2x+ 1 . Since x G [2* 1 ,2*- 1] it follows that y G [2\2* +1 - 1]. The equivalent of equation (7) is: 

y = 2* + g, (76) 
with o; G [0,2*- 1). Also, the definition of y and equations (7) and (76) imply that: 

q = 2p+1 (77) 
This equation shows that q is an odd number, therefore its binary representation is given by 



, with q G {0,1}, V/G and Cq= 1. The binary representation of pis 



, therefore by using (77) one can write: 



« 
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z;>2'=i;>2-. 



(78) 



5 From (78) again it follows that c, = b M for any / e [1 , ]. Also, from (77) it follows ve that 



10* 



2 



By using these observations and by replacing the expression of the "tail" Tgiven in equation (75), one can write equa- 
tions (73) and (74) as follows: 



15 



20 



*{H(HUHF x m DX 



(79) 



and 



25 



30 



+[G(//(..(ArF; w ^) e ,) tw ..) e ,) q ]<,» 



(80) 



where y(m) are defined as 



35 



j(m) = 4 



V 



+ 1 
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for any mS [1,AJ. 

Equations (79), (80) are equivalent with equations (25) for the odd values of x (odd-numbered subbands) Hence the 
propositions P R (k +1) are true in this case too. By joining equations (65), (66) with (79), (80) we derive the set of 
propositions P R (k+ 1) of level k + 1, for any values of x. One can also verify immediately that the filters calculated by 
equations (55) - (58), (69) - (71 ) can be derived from the general description of equations (9) - (1 6) for every / G [0,2*" 1 
-1J, if in these equations k is replaced with k+ 1. Hence, the derivations of the filters for level k + 1 are true. 
This leads to the conclusion that the propositions P(fr+1 ) are true for the case of £=frf 1 . This means that they are true 
for every E, . 

Properties of the Prediction Filters for Biorthogonal Point-symmetric Filter-pairs 

[0037] Several symmetry properties of the prediction filters derived in the previous subsection for an arbitrary de- 
composition level /care demonstrated. The general form of the prediction filters for level k is given in equations (9) - 
(16). The prediction filters of level /r+ 1 are obtained from (9) - (16) by replacing frwith k+ 1 : 



55 



(81) 
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r** 1 V 1 c 1 c* 



(82) 



cJf* 1 -1 r 1 

F8 /+2 =Z P 2 F 4/,V 



(83) 



(84) 



*0« 



^3 /+4= ^4/1 +F 0 / V/,0' 



(85) 



75 



r 8 /+5 ~ r 1 r 4/ t o- 



(86) 



(87) 



20 



/+7 =/= 4/,1 + F 3^i/,0» 



(88) 



25 



where /= 0,1 ,..,2*" 1 - 1. 

The mathematical foimalism describing the symmetry properties proven in this section is expressed for biorthogonal 
point-symmetric filters by the set of propositions given below: 



30 



Ps(*) = 



35 



1 ^W-*^^,^). 



(89) 
(90) 
(91) 

(92) 



40 



45 



form « 0,1 ,..,2* 2 -1 and*>1. 

These equations indicate the fact that we can derive half of the F-filters of level k as the time-inverses of the other half 
of the set of filters for the same level under by some shifts. Specifically, the filters are time-inversed in groups of four 
filters that lay in a "mirror" fashion in the group of the prediction filters. Thus, the first four F-filters are related with the 
last four, the second four F-filters with the penultimate four, and so on. 

[0038] These properties will be demonstrated by mathematical induction. Thus we first give the proof of P s (2). Then 
we assume that the equations P s (A) are true, and based on this assumption we derive the symmetry properties P s 
(fr+1) between the F-filters of level fc+1 . If the derived set of equations P s (K+1) can be simply obtained from P s (/c) by 
replacing k with fr+1 , then according to the induction principle the propositions P s (£) are true for any level E, E > 1 . 
Throughout the proofs given in this paper, the following relationships are used for the prediction filters of the first 
decomposition level: 



50 



fUz') = z'fUz), 



(93) 



F](z')=F](z), 



(94) 



55 



Fl(z')=z 2 Fl(z) t 



(95) 
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F 3 (z ) =z F 3 (2), 



(96) 



^o(z) = -Fl(z), 



(97) 



F 3 (z)=».zfJ(z 1 ). 



(98) 



10 



[0039] The symmetry propositions P s (2) corresponding to the case k = 2 are: 



15 



20 



P *(2) = 



F 0 1 (z) = zF 4 l (z-') > 


(99) 


F, 2 (z) = F 5 2 (z-'), 


(100) 


F l \z) = z 1 F>(z- 1 ), 


(101) 


F, 1 (z)=zF 7 2 (z- 1 ). 


(102) 



We assume that the symmetry propositions P s (K) are true and based on them we derive the symmetry relationships 
for level /r+1. Consider equation (89), which is true by assumption, and replace z with & and -z % respectively. The 
25 following two equations are obtained: 



30 



zV 



4(2 *., 



(z\ 



^^4(2*., m-l)(- z - 1> *)- 



(103) 



(104) 



35 The polyphase components of F * , with m = 0,1 ,..,2*- 2 - 1 are given by: 
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(z) = %z H [F^ m (2 J4 ) - F^-r*)]. 
In equations (105), (106) we can replace the terms F*^*), F* 4m (-.z**) by using (103), (104), yielding: 



W = nM)<* V^M^-**)] « F^Z) = ZF^., m . 1)0 (z 1 ), 



(105) 



(106) 



(107) 



(108) 



With m= 0,1 ,..,2* 2 - 1 and fc>1. 

Let us start by proving the symmetry properties for the filters of the form F* +1 , with m=0,1 ,..,2 fr1 - 1 ; these properties 
are separately derived for m even and m odd. For the case of m even, wVdeflne m = 2], with 2j = 0,2,4,..,2*- 1 - 2, 
equivalent to y'= 0,1 ,2,. .,2** -1 . Hence, from equations (81) and (98) we have: 

rfmW = ^M) = *S W tt ' SfI(z)FI u [2) « F^Jz) = F^ /0 (Z) + Fi(/ 1 )f* yi (z). (109) 
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Since y'= 0,1,.,2* 2 - 1, we can substitute (107) and (108) in equation (109), obtaining that: 
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4(2" '-MM' w 4(2' 

From (85) in which z is replaced with z* 1 and from the definition of m we obtain that: 

F*?Jz) = (111) 

for m = 0, 2, 4, .., 2* 1 - 2. 

For the case of m odd, we may define m =2 j '+ 1 with 2/+ 1 = 1 , 3, 5,.., 2*' 1 - 1, equivalent to /= 0,1 ,2,.,2*' 2 -1. Thus, 
from equations (85) and (98) we have: 

F^ 1 m (z)= F^ 1 M (z)= /^ y i (z) + F^(z)/< yo (z) « /^4 1 m (z)= /< yi (z) - zF^(/ 1 )/< /0 (z). (112) 
Since /= 0,1,.. ,2** 2 • 1, we cah substitute (107) and (108) in equation (112) and we get: 

f*: 1 ^) = ^(< {2 *.i j. 1) . 0 (^ 1 )-^3(^ 1 )< (2 *-i./. 1)i1 (^ 1 ))- (us) 

From (81) in which zis replaced with z" 1 and from the definition of m we obtain that: 



for m= 1,3,5,.,2*" 1 - 1. Thus combining the results of equations (111) and (114), we have: 



(114) 



^4 1 mU) = ^ 2 *.m.i)(^ 1 ) for m = 0.1 .2,.,2*" 1 - 1 . (1 15) 



To prove the symmetry properties for the prediction filters of the form , with m- 0,1, .,2* 1 - 1 , we follow the same 
rationale as before. Hence, these properties are separately derived 1of% even and m odd. For the case of m even, 
55 we denote m = 2y with 2/= 0,2,4, .,2* 1 - 2 equivalent to /= 0,1 ,2,., 2* 2 - 1 . By using equations (82) and (94) we obtain: 

rtViM" ^iM^iM « ^ m+ iW = z 1 F](z 1 )fJ yi (z). (116) 
Since /= 0,1 ,.,2* 2 -1 , we can substitute (108) in equation (116), obtaining: 

f*: 1 m+1 (z)=F;(z , )^ 2 , 1>1)0 (z 1 ) 

45 

From (86) in which z is replaced with z- 1 and from the definition of m t we obtain that: 

50 

form=0,2,4„. t 2*-i -2. 

For the case of m odd, we denote m= 2j + 1 with 2/f1=1,3,5,.,2*- 1 - 1 equivalent to /= 0,1, 2,. ,2*' 2 - 1. Thus, from 
equations (87) and (94) we obtain: 

55 ^^W^'asW^Iw^oW ~ /^ 1 m+l (z)= ^(z- 1 )^^. (118) 

Since /= 0,1,. ,2** - 1, we can substitute (107) in equation (118), obtaining: 



23 



* 
« 



EP 1 294 196 A2 



^^W-aFjtf^/^^W. (119) 



Mr 

From (82) in which z is replaced with z 1 and from the definition of m, we obtain that: 



5 



^ 1 m + ,<*) = ^ W* 1 ) ~ ^ 1 m + iO - *W-nMWl<A < 120 > 



8(2 v ' m+H*/ 4(2 -"M}+1 

for m= 1,3,5,..,2*" 1 - 1. Thus combining the results of equations (117) and (120), we have: 



«m W - 0-*D*i (/1) f ° r ^ = 0,1 ,2 '"' 2>r1 " 1 * (121) 

For the prediction filters of the form F M , with m= 0,1,.. ,2 /e * 1 -1 , we follow the same reasoning as before, hence the 
,s symmetry properties are derived sep&PStlly for m odd or even. For the case of m even, we denote m= 2/ with 2j- 
0,2,4,. ..2** 1 - 2 equivalent to y = 0,1 ,2,.. ,2* 2 - 1 , and we use equations (83) and (95), obtaining: 
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• 

rt'^W = f^s U2 (z) = z 1 /^(z)/< yi (z) o F*? m+2 (z) = zF^z 1 )/^^). (122) 



Since /= 0,1 1 ..,2*' 2 -1, we substitute (108) in equation (122) and we get: 



^ 1 m +2 (^)=^^(^)< (2 M.y. 1) .o(^ 1 ) 
From (87) in which zis replaced with r 1 and from the definition of m, we obtain that: 

form =0,2,4,. .,2^ -2. 

For the case of m odd, we denote m= 2y+ 1 with 2/+ 1 = 1,3,5,.. ,2* 1 -1 equivalent to /= 0,1, 2,. .,2*- 2 -I.Thus, for this 
case we use equations (87) and (95), and we get: 

^imaW- ^/.aW- ^M^M ~ ^i^W- ^^(z 1 )F* yo (z). (124) 
Since / = 0,1 ,..,2* -2 -1 , we substitute (1 07) in equation (124) and we obtain that: 



From (83) in which z is replaced with z" 1 and from the definition of m, we obtain: 



& W = ^F*; 1 2 *.i j. (z) » m+2 (z) = tf*l* (z* ), (1 25) 



8(2 •/•1)+2 V '-' * m +2 \*/ *■ ■ 4(2 . m -1)+2 

for m = 1 ,3,5,.. ,2*1 - 1. Thus combining the results of equations (123) and (125) we have: 

.m.i> + 2( Z * 1 ) for '" = 0»1.2,..,2' r1 -1. (126) 

Finally, for the filters of the form F , with m= 0,1 ,..,2*" 1 - 1 we follow the same approach as previously. For the case 
of m even, we denote m = 2/ with 4 /=% f 1 ,2,..,2* 2 -1 . Hence, for this case we use equations (84) and (96): 
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Since /= 0,1 ,..,2* 2 -1 , we substitute (107) and (108) in equation (127) and we get: 



4<2 4<2 

From (88) in which zis replaced with r 1 and from the definition of m, we obtain: 



rt^M - <(2^.A1H7(^ 1 ) ~ rt^W = ^O-d/'- ■ (128) 

form = 0,2,4,..,2*- 1 -2. 

For the case of m odd, we consider m = 2/ + 1 with /= 0,1 ,2,.., 2* 2 -1 . Hence, by equations (88) and (96): 
Since /= 0,1,..,2* 2 - 1, we can substitute (107) and (108) in equation (129) and we get: 
From (84) in which zis replaced with z % and from the definition of m t we obtain: 

25 rtn^W = ^*- 1 .y.iK3^ 1 ) ~ ^n^M = ^^W^' ( 130 > 

for m = 1 ,3,5,. .,2*" 1 - 1 . Thus, by combining the results of equations (128) and (130) we have: 



FK\^(z) = ^V./n-W 7 " 1 ) for m = 0.1 A.-^ M • 1- (131) 

Equations (115), (121), (126), (131) can be derived from equations (89) - (92) by replacing k with k + 1 Thus the 
symmetry propositions P s (k + 1) are true. This end the proof of the induction, that is the symmetry propositions P s 
(£) are true for any level E, with E > 1 . 

[0040] As a practical example of the symmetry properties, the filter taps of the prediction filters of the 9/7 filter-pair 
for decomposition levels 1,2,3 are shown in Table II, III and IV. These properties, expressed by the set of equations 
(89) - (92), allow the reduction of the necessary multiplications for the complete-to-overcomplete transform derivation. 
This complexity analysis is given below. 

[0041] As explained above, the !evel-by-levei calculation of the overcomplete representation from the critically-sam- 
pled pyramid can be performed by using two techniques: the LL-LBS method described above and the prediction-filters 
method in accordance with the present invention. Each technique operates in two modes depending on the current 
decomposition level. The first mode is the futl-overcomplete (FO) transform-production mode, where the current de- 
composition level Is the coarsest-resolution representation, therefore the low and the high-frequency subbands are 
produced (level 3 in the particular example of Fig. 3). The second is the high-frequency overcomplete (HFO) transform- 
production mode, where the current decomposition level is an intermediate-resolution level, and as a consequence, 
only the high-frequency subbands of this resolution need to be computed. In order to estimate the complexity of each 
technique, two factors are taken into account. The first is the necessary number of multiplication operations, which 
corresponds to the computational complexity of each method, since multiplication is the dominant operation in convo- 
lution^ systems. The second is the delay for the production of the results of every level, which can be estimated under 
some assumptions concerning the degree of parallelism achievable in the implementation of every technique. Since 
systems with a high degree of parallelism are assumed and designs with the minimum amount of multiplications are 
chosen for both methods, the results of this section are more realistic for custom-hardware rather than processor- 
based solutions; in the latter, a lower level of parallelism is feasible and the minimization of MAC operations is the . 
critical issue in the complexity reduction. 

[0042] The following section focuses on the complexity analysis of the 1 -D application of both methods. Moreover, 
following the extension of the prediction-filters to 2-0 decompositions, it can be shown that the level-by-level application 
of the 2-0 prediction filters is completely separable to the row and column filtering with the corresponding 1 -D kernels. 
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This matter is further discussed below. 



Required Number of Multiplication Operations 



[0043] We assume the general case of a biorthogonal filter-pair with T G denoting the number of taps for the filters 
H,<5 , and G,/* respectively. In addition we restrict our further presention to the case of point-symmetric filters, though 
the invention is not limited thereto; this option is mainly motivated by the fact that this sub-family of biorthogonal filters 
typically gives very good results in still-image and video coding. 

[0044] For both methods we assume convolutional implementations. A lifting implementation is possible for the pre- 
diction filters, if the factorization algorithm of Sweldens is applied. Assuming convolutional implementation, for every 
application of the filter kernels we need to perform 



15 



20 



and 



multiplications because of the point-symmetry property of filters H, G. Notice that, both for the LL-LBS and the prediction- 
filters method, the classical point-symmetric extension (mirroring) is assumed for the signal edges. For an one-level 
wavelet decomposition or reconstruction of an N - point signal, the required number of multiplications is: 



25 



r 



2 2 



^ 



N 



(132) 



30 



where X^/v^cf/V) denote the number of multiplications needed to perform a complete decomposition and recon- 
struction of A/samples respectively. For a decomposition where only the low-frequency (average) subband is produced, 
the required number of multiplications for an N -sample signal is: 



35 



T H+ 1 



2 



(133) 



40 



Similarly, for a decomposition where only the high-frequency (detail) subband is produced, the number of multiplications 



is: 



45 



Finally, for a reconstruction from the low-frequency (average) subband only (only the number of non-zero multiplication 
operations is taken into account), the required multiplications for an N -sample output signal are: 



50 



(135) 
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By utilizing equations (132)*(135), we can calculate the required number of muftiplications needed to derive the sub- 
bands A k , starting from subbands A* and d£, with k and / G [1,2*-1] denoting the decomposition level and the 
subband index respectively. As explained before, in order to ensure spatial scalability, all higher-resolution levels are 
assumed non-present during this procedure, hence d' q = 0 for any / 6 [1 , k- 1 ]. 

[0045] As described above k inverse transforms should be performed to reconstruct the low-frequency subband (or 
the input signal) in the case of the LL-LBS method; for an example of k= 3. Then, by performing a number of forward 
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transforms retaining the even or odd polyphase components, all the subbands of the overcomplete representation at 
level /r'are produced. In general, starting from the subbands of the critically-sampled decomposition of an Appoint 
sequence at level k, the number of multiplications required to perform the inverse transforms is: 



(136) 



10. 



The multiplications required to perform the forward transforms, when operating in 



the FO-mode, are: 



15 



(2 - !)X, (N) + (4 - 1)X, (f ) + .. + (2 4 "' - l)X, (#) + (2* - 1)X C (^) = 



(A-l + 2-*) + (l-2-*)[^tij 



(137) 



20 



25 



The total number of multiplications performed with the LL-LBS method in order to 
calculate all the subbands of level k\n the FO-mode can be derived by the sum of equations (136) and (137), given by: 



^LL-LBS.FO (*0 ~ N 



(A-l+2'-') 



T e +l 



(2-2-)]. 



(138) 



30 



When operating in the HFO-mode, the number of multiplications for the forward 



transforms of the current decomposition level k is modified, since for this level only the high-frequency subbands are 
35 produced: 



40 



(2 -i)x A (N)+(4~ DX, (f ) + .. + (2*-' - 1)X, (^)+ (2* - l)X D (£■) = 



= iV 



T„+l 



(*-2+2 1 - i )+(l-2-*)|^^_LL 



(139) 



45 Hence, the total multiplication budget is in this case: 
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LL-LBSJiFO 



(*) = Af 



T H + l 



(*-2+3-2-*)+ (2-2'-') 



(140) 
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In order to derive for the prediction-filters method the necessary number of multiplications for the level-by-level calcu- 
lation of the overcomplete representation in the FO and HFO modes, we need to specify the necessary convolutions. 
For the level-by-level construction of level k with k z 2, we only have the subbands A k ,a < , since D 1 = 0 for any /e [1 , 
As a consequence, the sets of equations (6) and (8) are simplified to the followftig expressions: 
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D;[n] = Fl[n)*4[n) + Fl[n]*DZ[n)' 



with k>2 t and 




(142) 



with i L G [0,2" -1], for every / G [1 , k - 2] and k> 3 . Also, 



(143) 



with / R e [0,2* 2 - 1], /r e Z and k 2> 2. 

[0046] Equations (141 ), (142) and (143) represent the calculation of the "left-half anjj "right-hair of the overcomplete 
pyramid as seen from Figures. Notice that each of the two subbands <4*,0* contains coefficients. From (141), (142), 
(143), it can be noticed that in the level-by-level framework, the calculation of the Overcomplete subbands with the 
prediction filters described by the sets of equations (6) and (8) are reduced to a compact form that depends only on 
the critically-sampled subbands of current level, and on the set of the filters F 1 with / G [1 ,/c], / G Z and m G [Q,2 M - 
1], mG Z. As an example, by replacing k = 3one can extract from (141), (1427and (143) the necessary convolutions 
for the particular example of the level-by-level derivation of the subbands of level 3 via the prediction-filters method. 
This particular example is illustrated in Figure 4. 

[0047] The total calculation budget can be reduced if instead of performing directly the convolutions in (142) and 
(143) with the filters F ,+1 , / G [1,/r - 1], /= 0,1 ,..,2** -1, one replaces their expressions with their equivalent form 

3/+{0,..,7} 

given in equations (9)-(16). From these equations, it can be seen that the direct convolution with the filters F M 

8/-H0....7) 

can be replaced by convolutions with the filters F M , F M , coupled with the reuse of the results of convolution with 

4/.1 4/.1 

the filters F \,f\ This means that an update structure can be implemented so that the initial convolutions per- 

formed with the filters F t F] ^ .f\ in the calculation of the subbands A k 4 ,tf given by (141 ) are reused in the calculations 

0 2 3 11 

performed in (142) and (143). This observation is formalized in Table V, where for each of the equations (9)-(16), the 
necessary convolutions and the corresponding number of multiplications are given as a function of the number of input 

samples (/v), the decomposition level (fc) and the number of taps of the used filter, denoted by T F " 1 with q = 0,1. 

4',<7 

Notice from Table V that the reused convolutions of equations (9)-(16) are: 



[*SM*4M]. [^WflfW], [^iW*4!W] 



and 
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w 



[/?m* asm] 



. Notice also that 



[/*M*4fM] 



is equivalent to 



-[l?W*4!W] 



, as it comes out from equations (1 7), (1 8), and hence can be used also for the convolution F* [n]M*[n] which appears 
15 in (9). 

[0048] By summing up the multiplications reported in Table V and the necessary multiplications required for the filters 
of the first level, one can derive the number of the multiplication operations needed to calculate in the FO mode the 
subbands for every level klof an Appoint input signal: 



20 



M 



P-FilterfcFO 



N_ 
2 k 
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k 2'-'-l 
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/-2 i-0 i-0 J 



2 k 



( k 2*" 3 -l 



v f-2 f-0 



25 in the HFO mode, where only the high-frequency subbands of the overcomplete transform are calculated, only the 
mers F J!\> F J?\' F ^\> F !>\ andF a/\2' F S /\3' F ai\r F fl/* + 7 V/ l ,/r have to be used forthe"left-hair and "right- 

half parts, as seen from (142) and (143). For every value of k, this corresponds to filters F * ,F * and F ,F 

of equations (11), (12) and (15), (16). Thus the situation is limited to the second and the fourth row of Table V and to 
filters F 1 and F 1 for the first decomposition level (equation (141)). As a result, if one assumes that the system is 

2 3 

operating in the HFO mode, equation (144) is modified to: 
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f k -i 



N 



(145) 



For the prediction filters of level 1 , one can reduce the number of multiplications by exploiting the point-symmetry (or 
half-point symmetry) properties of these filters, expressed in equations (93)-(98). In addition, the general symmetry- 
properties for the prediction filters of point-symmetric biorthogonal filter-pairs can be used to reduce the number of 
multiplications for the higher decomposition levels as well, as explained in the following. 

Convolving an input sequence l[n] with two FIR filters L{n) and Hfn] that are linked with the relationship W[z) = r a U 
(ir 1 ) requires the same multiplications but performed in a different order This is a corollary from the fact that by rep- 
resenting the two filters in the frequency domain, one has W(eP) = eA^e^)*, and hence the sequences U{e&)-I 
(e^), U(^y'l(e/°) represent the same multiplications but with alternating signs for the imaginary part (due to the 
complex conjugate In the frequency domain). As a result, a memory structure can be utilized to delay the intermediate 
results of the convolution in the time domain for the parallel calculation of U[n] * {n] and * i[n). For an L -tap filter 
U[n], such a memory structure is shown in Fig. 6. Similar forms are commonly utilized for the efficient realization of 
FIR orthogonal filter-banks. Using such memory structures for the parallel application of the prediction filters, the re- 
quired number of multiplications can be reduced by half, for every set of prediction filters F. This can be seen from 

the example of Fig. 4, where, if four memory structures are utilized for the application of filters f 2 , F^.f 2 , F 2 to the 

Input sequences A* t ri*, the results of the application of filters F 2 , F 2 , F 2 , F 2 are obtained as well, due to the symmetry 
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properties; see equations (99)-(102). Hence, subbands A*D* and A*D* are create in parallel by applying only one 
quadruplet of filters impiemented with four memory structures similar to the one seen in Fig. 6. In the same fashion, 
the application of filters >f\,f\A,l\ in Figure 4 creates the results of F 3 ,F 3 ,F 3 ,F 3 .F 3 ,/^ 3 ,F 3 

0 1 2 3 4 5 6 7 ' 12 13 14 15 8 9 10 11 

respectively; see equations (89)-(92) with = 3, Generalizing this property, for every level k, the convolutions of filters 

F \> F \> F \K and filters/^ or F with /G [2, /c] and V/:/e[0,2'-1] with the input subbands a'd 1 suffice for the calculation 
0 12 3 / 2 +y 0 o 

of all the subbands of the overcbmplete representation of this level. Equivalently, a mixed calculation-order can be 

I i I i ' 
selected, i.e. the convolutions with filters F , F , F , F can be used to produce filters F „« 

4m 4/71+1 4m+2 4/n+3 4(2 -m-1) 

/ / / 

F j , F ! , F r1 respectively for the even values of m (and zero) and vice-versa for the odd 

4(2 -m-1H1 4(2 -m-1)+2 4(2 -m-1H3 

values of m, with m=0,1 ,..,2 / " 2 -1 and / e [2,/c]. For example, for the calculation of the overcomplete transform of level 
3, the convolutions with filters f\,f\,f\,f\ and f\,F* t F* ,F 3 can provide the results of filtering with F 3 ,F 3 ,F 3 t F 3 

0 1 2 3 8 9 10 11 r 3 12 13 14 15 

and F\,f\,F*,F* respectively.. 

4 5 6 7 

[0049] However, as mentioned before, in order to reuse the previously produced results, the implementation of the 
convolutions with filters /=* and F ' is based on the update structure shown in Table V and not on the direct application 

/ 2 +/ 

' M 

of the filters. Thus, filtering with F and F is performed by the separate applications of filters F (z),F M (z), as 

' 2V 4/ |0 47.1 

seen in Table V. As a result, in order for the multiplication-reductions to be valid in this update-structure implementation, 
it must be shown that all convolutions with the filters seen in Table V can produce the convolutions with the time-inverse 
filters as well, without additional multiplications. This is shown in the next paragraph. 

[0050] By replacing z with in equations (9)-(16), the time-inversed update-structure of the prediction filters can 
be written: 

r£{*-'hr%{*-')-zFl(:- l )FX{z- x ), (146) (147) 



^(•-J-^M/^M, (148) ^ J (z-) = F^(z-') + ^ , (^K M - , ( 2 -') (149) 



r^{*-') = F!('- l )F£(z-% (152) ^ 7 (r-') = F^(z-«) + F J '(,-')^ r '(z-') (153) 

For the filters seen In the right parts of expressions (146)-(153), the application of r a F M (r^)-L, z a F" (r 1 Wo, with 

4/.0 4/.1 * 

a = 0,-1 , to any inputs / 2 can be performed by F M (t) l v F M (*)/ 2 with the use of two memory structures such as 

4/.0 4/,1 

the one seen in Figure 6. In addition, based on the symmetry properties shown in equations (93)-(97), any convolution 
with filters F* (^ 1 ),fJ(z- 1 ), (z-*), F* (z-') is equivalent to the convolution with ^, /=J , F^, F* (with the appropriate 

delays). As a result, all the filter-applications shown in the time-inversed update-structure of (146)-(153) can be imple- 
mented by the convolutions shown in Table V and a number of memory structures such as the one shown in Fig. 6; 
hence the update-structure implementation can provide the convolutions with the time-inversed filters with no additional 
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arithmetic operations. Summarizing, since the update-structure implementation of filters /^(z) can provide the results 
of filters /^(z -1 ), based on the prediction-filter symmetries, the results of filters ^(r 1 ) provide the convolutions with 

[0051] Exploiting this symmetry by selecting the filters to implement according to the mixed calculation-order that 
was mentioned before, the number of multiplications in the FO-mode with the prediction-filters method is reduced to 
the half of Table V. By choosing to implement equations (9)-(12), the required multiplications are: 



P-FiJtcrwymJFO 



N_ 
2* 



(154) 



Similarly, for the HFO-mode, the implementation of equations (11), (12) requires: 
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r * +1 



7"., +1 



(155) 



Table VI shows for both the FO and the HFO-modes, the number of taps for the filters f'" 1 , F M , F M for the typical 

25 4/ 4/.0 4/.1 

9/7 biorthogonal filter-pair, with /G[2,4]. This table reports also the required number of multiplications for the LL-LBS 
method and for the prediction-filters method for the construction of all subbands of level I as calculated by equations 
(138), (140) and (154), (155) respectively. The last column of Table VI shows the percentage of the reduction in the 
multiplication budget. 

30 [0052] The numbers of multiplications shown in parenthesis correspond to an approximation of the prediction filters 
where, for the filters of the update structure of every level, all taps smaller than a threshold are set to zero, In this way, 
the size of the filters of each level is reduced, while a good approximation of the final result is obtained. This technique 
cannot be applied in the LL-LBS approach since the taps of the biorthogonal filter-pairs do not have magnitudes below 
the chosen thresholds. Table VIII shows the values used for the thresholds and the resulting maximum mean-square 

35 error (MMSE) between the results obtained with the original and threshoided prediction filters when applied in the 2-D 
(row-column) manner to the 8-bit images of the JPEG-2000 test-set. It can be observed from the MMSE values that 
the chosen method for thresholding has a minimal effect on the results of the prediction filters while it reduces signif- 
icantly the computational load. 



40 Delay for the calculation of the subbands of decomposition level k. 

« 

[0053] The delay occurring in the calculation of the subbands of the overcomplete pyramid of decomposition level k 
using the prediction-filters and the LL-LBS methods is now presented. As in the previous sections, we discuss the case 
of biorthogonal, point-symmetric filter-pairs. Consider that the two methods are implemented in a system where one 

45 application of a filter-kernel on an input sequence requires a^g processing cycles for the LL-LBS method and a PF 
processing cycles for the prediction -filters method. Furthermore, to diminish the side effects of scheduling algorithms 
for the multiple filtering operations, we assume the case of high parallelism, where one filter-kernel per required con- 
volution is present. In this way, every filtering application initiates as soon as sufficient input is present. Moreover, to 
facilitate the description, the delay resulting from the storage or retrieval of intermediate results is nottaken into account. 

so In addition, in this case of high-parallelism, the delay for the calculations in the FO mode is equivalent to the one for 
the HFO mode, with the latter requiring less filter-kernels. Hence no distinction is made between the two modes in the 
following, and the numbers for the filter-kernels refer to the FO mode, which represents the worst case between the two . 

[0054] Starting from the subbands A*d£, the LL-LBS method performs k inverse transforms and I k (2'-1 ) forward 

0 0 r fe1 

55 transforms to produce the subbands of the overcomplete representation. See Figure 3 for an example with k = 3 and 
Fig. 5 which represents the overcomplete pyramid in the general case of k + 1 decomposition levels. The cascade 
initiation of each of the inverse transforms at levels k - 1 , k - 2,..,1 requires 
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input samples to be present at every level, so that mirroring and the first filter-kernel application can take place. Such 
an example is given in the top-left part of Fig. 7 for the calculation of level k - 1 from level k. Notice from this figure that 
since the subbands D 1 with /G [1 M- 1] are not available, the samples of the low-frequency subbands are interpolated 
with zeros at every level /. 

[0055] After the initiation phase, all filter-kernels of every level work in parallel to perform the inverse transforms; this 
is equivalent to the inverse RPA algorithm of Vlshwanath, but without considering the high-frequency subbands. In 
order to produce 



15 



samples at every level, the filter-kernel of the previous level must be applied 



20 



times, as seen in the top-left of Figure 7 with T G = 7. Similarly, for the initiation of the entire forward transforms, 



25 



input samples must be present in the signal X and at the levels 1 ,2,.., . This implies 



30 



35 



applications of the filter-kernel in the previous level, to initiate the forward transform in the current level. An example 
of the mirroring and initiation of the forward transform of level 1 which is calculated from the signal X is given in the 
lower-left part of Fig. 7 with T H = 9. After this point, all filter-kernels of the LL-LBS system work in parallel. Hence, the 
latency occurring in the initiation phase of the production of all subbands of level k is: 



40 



w =[ (t -,)[l*bj +i [Mij] 



<*ms- (156) 
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50 



55 



The total time required for the completion of all subbands is determined by the filtering of the sequence with the max- 
imum length, since during this process the highest number of consecutive filter- kern el applications occurs. The se- 
quence with the maximum length in the pyramid of the overcompiete transform is the sequence of the reconstructed 
signal X. The filtering of X requires j applications of filter Hto produce the subband /C , as it can be noticed from the 
pictorial explanation given in the lower part of Fig. 7. 

[0056] To summarize, the latency occurring in the production of the first coefficient of this sequence is 



Has* 



'lbs 



processing cycles (see equation (156)). The filtering of X requires ja^s processing cycles to produce the subband 
A . After the completion of this operation, finalized with the mirroring of the subband border, the coefficient 
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a 



1 



s is produced. The remaining filtering applications for the following decomposition levels (levels 1 , 2, .... *) require 



'lbs 



processing cycles, so that, in the same manner, all the coefficients 



15 



1 

a N 



with / e [2, /c] are produced. As a result, the total delay of the LL-LBS system for the production of all the subbands of 
decomposition level /c for an A/ -point input sequence is: 



20 



A^«(*^)-[(*-o[ i ^ t j+7+* 



fl LBS = 



Anil 



.LL-LBS (^) + 2 



\ 



l LBS 



(157) 



25 



For the prediction-filters method, the convolutions with the filters can be initiated in parallel for the calcu- 

k ~k 0 12 3 

lation of the subbands A , Dr. Subsequently, the filters that produce the rest of the subbands of level k can also be 
applied in parallel using ihe update-structure implementation. The application of all these filter- kernels will occur as 
soon as enough coefficients from the sequences 
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50 



are calculated, so that the necessary mirroring in the update structure can be performed. In addition, a delay equal to 
the maximum filter-length is required so that the calculations of the subbands that are produced by the time-inversed 
impulse responses are Initiated as well by the memory structures. Hence, the latency for the initiation of all the con- 
volutions is: 



55 



max {7^ ,r, ,r_,} + l 



+ max{T, ,T, ,T^} 



, (158) 
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with /e[2>1],/E(0,2* 2 -1]je[0,3]. 

Then, the delay for the completion of the process is simply: 
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(159) 
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20 



The results of (1 57), (1 59) show that, under the same assumptions of system parallelism, the complete to overcomplete 
transform derivation of resolution level k for an N - point input sequence via the prediction-filters method achieves a 

N 

delay proportional to -^app processing cycles, while the LL-LBS approach produces the subbands of the same level 

with a fixed delay proportional to -a LBS . Table VII shows a comparison for a typical case where A/= 512, T H = 9, T G = 

7 for various decomposition levels. We notice from these results that a significant gain is achieved in the reduction of 
the system delay with the prediction-filters method. 

Extension in 2-D 
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[0057] The 2-D extension of the LL-LBS and the prediction-filters method can be intuitively performed by extending 
the 1-D application to a separate application to the rows of the input subbands and to the columns of the produced 
results. Such extensions are due to the separability property of the DWT. For the level-by-level construction of the 
overcomplete DWT of an N x M input frame, the separable row and column application leads to the same computational 
gains for the prediction -filters method in comparison to the LL-LBS approach as in the 1-D case. The delay of every 
method when applied in two dimensions is dependent on the level of parallelism for each system. Since the prediction 
filters method requires only the application of a set of filters in the row and column directions, under any practically- 
applicable level of parallelism, this method is expected to achieve a much lower delay for the production of the 2-D 
overcomplete transform than the 2-D LL-LBS method, which consists of a cascaded multirate filter-bank implementa- 
tion. Similarly, every method can be implemented by two separate systems: one for the row and one for the column 
processing. If the results after the production of one row of every subband are immediately used for the continuation 
of the vertical filtering for ail the columns of this subband, and the column filtering completes when the next row is 
received, the delay for the production of the results is simply the delay for the completion of the row-by-row filtering 
plus the delay for the completion of the columns' processing with the results of the last row. The column processing 
begins after an initiation latency so that enough coefficients exist columnwise for the mirroring and for the initiation of 
the filter-applications required for every method. Since the process is separable, the initiation latency is equivalent to 
the one of the 1 -D case. Hence, for the LL-LBS method, the required processing-cycles are: 
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while the prediction-filters method requires: 
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The results of (1 60), (1 61 ) show that the predict! on -filters method achieves a delay proportional to — _a PF processing 

cycles for the complete to overcomplete transform derivation of resolution level k for an N x M Input frame, while the 
LL-LBS approach achieves a fixed delay proportional to ^—la^, under the same assumptions of system parallelism. 

[0058] From the above, the skilled person appreciates certain aspects of the present invention as it relates to the 
complete to overcomplete DWT derivation. For the level-by- 1 eve I calculation of the overcomplete DWT for biorthogonal 
point-symmetric filter-pairs, the required computational-budget is reduced by creating a structure that exploits the sym- 
metries of this new approach. A complexity analysis reveals that the proposed prediction-filters scheme in accordance 
with the present invention is more efficient than the conventional method for the complete to overcomplete transform 
derivation in a scalable video coding system because it offers computational savings and a scalable reduction in the 
system delay. These features lead to inherent computational scalability for the prediction-filters, since, under fixed 
delay-constraints for the performance of the complete-to-overcomplete transform for every resolution level, the codecs 
that work in resolution-level k can be implemented with filter-kernels that operate in 2 2 * 2 times smaller clock-frequency 
than the kernels used in the system that operates in the full-resolution level {k = 1). Conversely, using the same im- 
plementation technology for all resolution levels, in the low-resolution codecs the level of parallelism can be reduced 
by well-known techniques such as folding or retiming without surpassing the delay-constraint of the full-resolution 
codec. This in turn leads to simple and more power-efficient. The scalability in the power consumption can be a critical 
point in the total system design since in many cases very-low resolution video-codecs are designed for portable devices. 
[0059] From the flexibility point of view, by thresholding the filter-taps of the update structure implementation, the 
number of multiplications required for the prediction filters in reduced without a substantial loss in the accuracy of the 
produced results, something that is not the case for the LL-LBS method. In addition, the prediction-filters system pos- 
sesses a simple implementation structure, since it consists of the parallel application of a set of filters to the same input 
(single-rate), while the LL-LBS approach requires precise control on the data-flow in the multirate implementation- 
structure in order to achieve a high-parallel system. As a result, the selective construction of specific blocks of the 
various subbands of the overcomplete transform in the decoder is expected to be much easier in the prediction -filters 
system than in the conventional LL-LBS approach. 

[0060] In the following implementations of the above methods are described. In further embodiments of the present 
invention a video encoder and/or decoder based on the in-band motion compensation theory (codec of Fig. 8), either 
in the spatial or wavelet domain, are disclosed. 

[0061] In a second embodiment the 'bottom-up' ODWT (overcomplete discrete wavelet transform), based on the in- 
band motion compensation methods described above, can be used in a wavelet video encoder in the following way 
(see Figs. 8 and 9). The wavelet video encoder has the in-band structure, which means that motion compensation 
(MC) performed in the motion compensation module 1 28 is performed in the wavelet domain. The motion vectors (MV) 
132, provided by a motion estimation (ME) algorithm (e.g. block-based) carried out in the motion estimation module 
130, are preferably given with highest possible accuracy and are either estimated in the spatial (see embodiment of 
Fig. 8) or in the wavelet domain (see embodiment of Fig. 9). The 'bottom-up' ODWT module 122 takes as input the 
reconstructed reference wavelet subband image after the inverse quantisation IQ In IQ module 111 and summation in 
summer 114. The overcomplete representation generated by application of a digital filter is stored in a memory buffer 
(Oref FM) 124. The motion compensation process carried out in the motion compensation module 128 takes as input 
the motion vectors and this overcomplete representation. Depending on the motion vectors, blocks of coefficients are 
selected using a selection means in the motion compensation unit 128 from the overcomplete representation. These 
coefficients are combined into a wavelet decomposition that is a prediction of the wavelet decomposition of the current 
image of the video sequence. This prediction is subtracted from the wavelet decomposition of the current image in 
subtracter 106 resulting in a wavelet subband error image. This subband error image is subsequently quantised in 
quantiser 107 and coded in subband coder 108 . The subband error image is also inversely quantised in IQ module 
1 1 1 and added to the previous prediction from the MC modu le 1 28 in adder 1 1 4 in order to obtain the next reconstructed 
reference wavelet subband image to be used by the 'bottom-up' ODWT module 122 in the following iteration.' 
[0062] The main difference between Fig. 8 and 9 is that in Fig. 8 the motion estimation is carried out in the spatial 
domain whereas in Fig. 9 these vectors are determined in the wavelet domain. In the embodiment of Fig. 8 the motion 
vectors are supplied to the motion compensation module and the receiver from the motion estimation module 130. A 
selection means in MC module 1 28 selects the most appropriate subbands, e.g. block subbands, from the overcomplete 
representation in buffer 124. In Fig. 9 the motion estimation is made using the subbands of the overcomplete repre- 
sentation stored in buffer 124. This motion cestimatlon can be made level-by-level. 

[0063] The wavelet video decoder (see Fig. 10, having modules common to both embodiments of Figs. 8 and 9) 
operates in an analogous way. The input for the subband decoder module 134 is the quantised and coded wavelet 
subband error image. The decoded error image is supplied to the inverse quantiser 135 which ouputs an inversely 
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quantised subband error image which is summated in the summer 138 with the predicted wavelet subband image 
originating from the motion compensation module 1 42. This results in the reconstruction of the current wavelet subband 
image of the video sequence, which will subsequently be reconstructed by the IDWT (inverse DWT) module 140. The 
reconstructed wavelet subband image is also the input for the 'bottom-up 1 ODWT module 148 that generates the over- 
complete representation to be stored into the memory buffer (Oref FM), e.g. using application of predictor digital filters 
as described above. Then the motion compensation module 142 selects wavelet coefficients from this overcomplete 
representation based on the decoded motion vectors 150. These wavelet coefficients are combined into a wavelet 
decomposition that is a prediction of the current wavelet subband image of the video sequence. This prediction will be 
summated to the decoded and inversely quantized wavelet subband error image in the following iteration in summer 
138. Note that the switches 116, 118; 152 in both the encoders of Fig. 8 and 9 and the decoder of Fig. 10, respectively 
provide either the intra (open switch) or the inter (closed switch) encoding/decoding state. The text above is a description 
of the inter encoding/decoding state. In the intra encoding/decoding state, a full wavelet subband image is processed 
instead of a subband error image. 

[0064] Alternatively formulated the proposed coding scheme with motion estimation in the wavelet domain can be 
applied to the encoder shown in Fig. 1a. The operation shows a hybrid coding structure; thus when coding in intra- 
frame mode, the current frame is wavelet decomposed (DWT module 2) and compressed with a block-based intra- 
band coder (SBC module 8). This intra-compressed frame is decompressed (SBD module 12) and the reconstructed 
wavelet decomposition is used by the CODWT module 22. This module 22 operates in a subband-by-subband manner 
and constructs from.the critically-sampled decomposition of every level, the overcomplete transform of that level, which 
contains all the subband information that is missing from the critically sampled pyramid due to the subsampling oper- 
ations of every level. An important feature of the invention is that for this operation, methods of the present invention 
provide fast calculation-algorithms with identical algorithmic performance to LBS. These calculations include digital 
filters. The overcomplete representation is used for in-band motion estimation since it is shift invariant. The motion 
compensation is performed in module 28 in the critically-sampled pyramid, which is subsequently subtracted from the 
current frame and coded (error frame) in the subtracter 6. The decoder operates in a similar manner as seen from the 
lower part of Fig. 1b. Thus in the proposed structure, there are transform -domain intra-frames and error-frames. The 
embeddedness of wavelet-based coding and the level-by-level operation for the motion estimation and compensation 
guarantees that drift-free scalability in the decoding can be achieved. To evaluate the performance of the proposed 
framework, some preliminary results are shown in Table IX for the first 48 frames of the football sequence (luminance 
component) for two different bit-rates. The compression was performed by using one l-frame (intra) and seven P- 
frames (inter); the target bit-rates of Table IX for the decompression were simply met by decompressing at fixed rates 
the intra and error frames. For example, for 760 Kbps, the l-frames were decompressed at 1 .0 bpp and the P-frames 
at 0.2 bpp. It can be seen that the proposed algorithm, even without any sophisticated rate-allocation mechanism, is 
competitive to the 3-D SPIHT algorithm of Kim and Peartman and outperforms MPEG-2. Figure 1 4 shows an example 
of frame 5 decompressed at various resolutions with the proposed scheme. It must be noted that for all the experiments, 
the sequence was compressed once and ail the presented results where achieved by simply decompressing to various 
bit-rates and resolutions. The conclusion is that the approach provides resolution, quality and temporal scalability in 
video compression with the hybridcoding structure if the in-band approach is adopted. The hybrid in-band (motion 
estimation and compensation in the transform domain) approach using overcomplete representations equipped with 
the fast calculation methods can decode to a variety of bit-rates and resolutions achieving at the same time very 
competitive performance to state of the art 3-D wavelet video-coding and coding standards, which are often proposed 
in the literature as the best solution for fine granularity scalability in video compression. 

[0065] In a third embodiment wavelet video encoders and decoder based on the in-band motion compensation theory 
exploiting the in-band motion compensation theory based prediction rules are disclosed. 

[0066] Instead of using the 'bottom-up' ODWT in the video encoder, the wavelet coefficients required by the motion 
compensation process can directly be calculated from the reference wavelet subband image without the need for 
calculating ail of the subbands of an overcomplete representation. Instead, only those subbands are calculated as 
required, whereby these subbands can belong to the set of subbands of the overcomplete representation. The prop- 
ositions in the in-band motion compensation theory provide the required motion compensating prediction rules that 
correspond to all possible translations. These prediction rules make use of prediction filters and of particular subbands 
of the reference wavelet subband image. Which prediction filters to use and which subbands to choose, is determined 
by the motion vectors. If the motion estimation process is block-based, then the prediction rules only take as input 
blocks of wavelet coefficients from the reference wavelet subband Image. Hence, this 'bottom-up' motion compensation 
process does not require the computation of a total overcomplete representation, but calculates the required wavelet 
coefficients directly on a block-by-block basis. Such a wavelet video encoder is shown schematically in Fig. 11 and the 
decoder in Fig. 12. The major difference between the embodiment of Fig. 11 and that of Fig. 9 is that the motion 
compensation module 128 uses the motion vector information obtained from the motion estimation module 130 to 
calculate only the best subbands for motion compensation. The method of calculation relies on digital predictor filters 



36 



EP1 294 196 A2 



as described above but avoids the requirement to calculate all the subbands of the overcomplete representation. In 
Fig. 11 cnotion estimation is performed in the wavelet domain, but it can also be performed in the spatial domain as in 
Fig. 8. 

[0067] In a fourth embodiment resolution scalable wavelet video encoders and decoders based on the in-band motion 
compensation theory are disclosed. 

[0068] The wavelet video codecs, disclosed in the first to third embodiments, are both capable of supporting resolution 
scalability while avoiding a drift between the encoder and the decoders of different resolution levels. Conversely, a 
spatial domain motion compensation encoder causes a drift between what is encoded at full resolution and what is 
being decoded at a lower resolution. The reason for this drift is that the encoder uses all information available in the 
full resolution image (all subbands), which is not all transmitted to the decoders. A solution would be to code every 
resolution level independently from each other so that ail possible configurations of decoders are supported, i.e. mul- 
ticasting. Multicasting is however very inefficient from the encoders point of view, because multiple encoded streams 
have to be generated. Also from a compression point of view, the efficiency is very low and a lot of redundancy between 
resolution levels remains. This redundancy can partially be removed by working in a layered fashion, which is the 
MPEG-4 and H.263+ approach. First a base layer, with lowest resolution, is encoded with a certain image quality. The 
result is subtracted frame by frame (after upsampling and interpolating) from a version of the video sequence with 
twice the resolution of the base layer. The resulting error frames form an enhancement layer and are coded (with or 
without motion compensation). Several enhancement layers can be generated in this way. Although this principle is 
an improvement of multicasting, a further improvement can be obtained by a hierarchical coding approach. This ap- 
proach requires a 'hierarchical* transform like the wavelet transform. The advantage of the wavelet transform is that it 
is critically sampled and supports multiple resolution scales. This is in contrast to the layered approach, where the total 
number of coded 'coefficients* is higher than the number of pixels of the full resolution image. Hence, an improved 
coding efficiency can be expected from the hierarchical coding approach. However, motion compensation should be 
performed in the transform domain to make full advantage of the hierarchical resolution property of the wavelet trans- 
form for video coding, hence the wavelet video codec structures of Figs. 8 and 11 , respectively are required. In order 
to exploit the resolution scalability property of the wavelet transform in the second and third embodiment coders, a 
simple precaution needs to be taken. When performing motion compensation on subbands (or subband blocks for the 
third embodiment coder) that are required to reconstruct a certain resolution level, subbands of lower wavelet levels 
should not be used in the motion compensation process (lower wavelet level corresponds to higher resolution level). 
For the 'bottom-up' ODWT this means that subbands of lower wavelet levels than the level of the subbands of which 
an overcomplete representation is generated, should not be used. The reason forthis is that a decoder for this particular 
resolution level, does not receive the information that corresponds to the subbands of the lower wavelet levels and 
hence can not use it. In practice this means that the propositions or prediction rules of the in-band motion compensation 
theory should be adjusted. Terms in these prediction rules, which correspond to lower wavelet level subbands, should 
be neglected to enable resolution scalability by the wavelet video encoders of the first two embodiments. 
[0069] In a fifth embodiment a method for wavelet-based in-band motion compensation and a digital, being useful 
for the first four embodiments is disclosed. 

[0070] A novel in-band motion compensation method for wavelet-based video coding is proposed, derived here for 
the 1 -D discrete wavelet transform but not limited thereto. The fundamental idea is the computation of the relationships 
between the subbands of the non-shifted input signal and the subbands corresponding to all translations of the input 
signal. The motion compensation method is formalised into prediction rules. 

[0071 ] The in-band motion compensation algorithm can be extended to 2-D images, decomposed using the separable 
2-D WT. The chosen approach is valid for N-level decompositions. The prediction rules of the algorithm allow the in- 
band motion compensation process to reach a zero prediction error for all translations of the image pixels. Besides 
the applicability of this 2-D in-band motion compensation algorithm for scalable video codecs, other applications are 
envisaged such as in-band motion estimation, overcomplete wavelet decompositions, etc. 

[0072] The discrete wavelet transform (WT) is inherently scalable in resolution and quality, which is a very much 
wanted property for streaming video e.g. on the Internet. A possible codec structure exploiting the WT for video coding 
is the in-band codec. In-band means that motion compensation is performed on the wavelet coefficients. However, the 
periodic translation-invariance of the critically sampled WT prohibits from reaching a zero prediction error on wavelet 
decomposition level J, if the shift or translation x of the image pixels does not belong to x = 2 J k t with k E Z In the 
following, we suppose that the translation of the image pixels is known. The novel motion compensation algorithm or 
method will be derived for the 1 -D WT. 

[0073] Concerning the concept of periodic translation invariance we denote by Y(z) the result of filtering the input 
signal X(z) with filter H(z). V(z) subsampled with factor Af= 2 is not invariant with respect to integer shifts or translations 
x G Zof X(z). This is caused by the subsampllng operator that retains, depending on x, either a shifted version of the 
even or the odd indexed samples of Y(z), which we denote by V^z) and V^z). In general, only a limited number 
of possible delayed outputs exist, equal to the subsampllng factor M. Therefore, this system is called periodic transla- 
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tion-invariant. Given an oven translation x = 2k with /r e Z, we obtain the following expression for the shifted, filtered 
and sufisampled signal Y 2 *^: 

Y*\z) = /[H 0 (z)X 0 {z) + H,(z)X,(z)] = /V tfVW ,(2), (186) 

with X^(2), X,(2) the polyphase components of X(2) and H 0 (z), H^z) the Type I polyphase components of H(z) [20]. 
This actually is translation-invariance, because the output sequence y 2 * (n) is a shifted version of sequence y^^n), 
obtained for t = 0. Conversely, for an odd translation x = 2k +1 the output signal is given by: 

V 2 ** 1 (z)=AH, (z)X 0 (z)+zH 0 (z)X, (z)]=2* ^2). (1 87) 



[0074] Now in-band motion compensation for 1 -D WT, Level 1 is derived. Depending on the translation x of the input 
15 signal, we obtain a shifted version of either Y 0ven ffl or V^z). Hence, motion compensation can be simplified to 
prediction of odd samples from even samples or vice versa. For the single level WT with filters H(2) and G(2), we 
denote respectively the average subband by A^^^z), and the detail subband by 0*^(2). We obtain the output signals 
*M ev8n (z), z k D gvef ji^ for x = 2k, and z k A od J i z) and 2*0^2) for x = 2k + 1 . Fig. 1 3 (level 1 ) represents a decomposition 
tree containing the subbands of the single level WT for different translations x. Without any loss of generality one can 
20 drop the delays 2* and one can reduce the in-band motion compensation problem to predicting the subbands 4^2) 
and 0^2) from the reference subbands A^^z) and D^, en {z). This results in the first prediction rule: 



• Translation x = 2k + 1 , level 1 : 



with 



P 0 (z) = D'\g,(z)G,(z)- zG 0 (z)G 0 {z)l 



P,(z) = D~\zH Q (z)G 0 (z) - H^(z)G^z)]. 



with 



P 2 (z) =D-\H,(z)G,(z) - zH Q (z)G Q (z)], 

45 P 3 (z) =D'\zH 0 (z)H Q (z)-H A (z)H,(z)l 

Proof: From the system of equations formed by the expressions for A wgn ($ and D^v^z) in polyphase notation 
(1 86), we can isolate the input signal polyphase components X 0 (z) and X^z): 

*° (Z) = Det H ^ ^^(^^^(^-^(^D^z)], (190) 
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with H^z) the analysis polyphase matrix [20] . Because the wavelet filters satisfy the perfect reconstruction con- 
dition, we necessarily have Det HJiz) = D [21 ], with D and q constants. To simplify the formulas, we consider 
q = 0. If we substitute the signal polyphase components into the polyphase expression (1 87) for D^z) we obtain 
(188). Similarly, if we substitute (190) and (191) into the expression (187) for A od Jiz) we obtain (189). 
• Similarly to the previous calculations, we have to determine the influence of the X(z) translation on the sub- 
bands of level 2. Consider the decomposition either of the subband z k A gven (z) or zM^z) as illustrated in Fig. 1 3. 
This is equivalent to a single level wavelet decomposition applied to the signals A mn {z) and A^z) which are 
shifted either by an even or odd translation k f or equivalent^ k = 2/ and k = 2/ + 1 with / e Z As a result, we obtain 
four alternative decompositions corresponding to the following translations x: x = 41, x = 41 + 2, x = 41 + 1 and t = 
4/+ 3, see Fig. 13. 

From a motion compensation point of view, the present invention predicts for all four translation types, the 
decomposition subbands starting from the reference subbands of the 2-levels WT, corresponding to x = 0: D wen 

A A 

(z), A wen (z) and D avon (z). The upper index indicates the decomposed subband of the lower level. If the trans- 
oven even 

lation is of type x - 4/, then we have translation-invariance. All other translation types x belong to the translation- 
variant category. The following prediction rules can be formulated: 
Translation x = 4/f 2, level 2: a a 

Apply prediction rule forx=2/c+1 to the subbands A even (z) and D * ven (z). 
Translation x = 4/+1 , level 2: fiVfln GVen 

A~4m A m% .. n A AXMU . P>dz)^everS z ^ 

A™ n (z) = Q 0 (z)A™ (2) + Q 1 (z)D™ (z) + A£ n (z), (192) 

with 

A„hh A mwmr , A mumn PJLz)®6van( z ) 

®even ( ®2 even ( Z)+ Q 3 (z)D™(z)+D e ; en (z), (193) 

with 

G 2 (z)=z- 1 P 0 (z)P 2|1 (z), Q 3 (z) = P 20 (z) +z'V 1 (z) P 21 (z). 
P 2 0 (z) and P2.1t 2 ) are the polyphase components of P 2 (z); (410) ™ n (z) /s the filtered and subsampled 

even 

signal, originating from P 3 (z)D even (z) by respectively filtering with H(z) or G(z), and retaining the even samples. 

Translation x = 41 + 3: analogous to t » 4/ + 1 . 

The level 3 part of the 1-D WT in-band compensation rules is now discussed. The following classification of 
translation parameter x on level 3 is found: x = 8 m, x = 8m + 4, x = 8m + 2, x = 8m + 6, x = 8m + 1 , x = 8m + 5, x 
= 8 m + 3 and x = 8m + 7 (Fig. 13). For the left branch of the decomposition tree of Fig. 13, i.e. the branch 
corresponding to even translations x = 2/c, we can apply the prediction rules from the previous section. However, 
for the right branch corresponding to odd translations x = 2k + 1 , we describe the in-band motion compensation 
relationships through new prediction rules. We formulate the prediction rule for shift x = 8m +1 . We refer to detailed 
description for the other prediction rules and the proofs. 
Translation x = 8m + 1 , level 3: 



«r {z)=Ro{z)A%sr (z)+ *i(*)z>5sr w lzJ +«f 



(194) 
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fl 2 W = / 1 WO 0l (z), 



[0075] The same approach can be followed as in the previous paragraphs to obtain prediction rules for higher de- 
composition levels. Furthermore, a recursive pattern is revealed when deriving the R-prediction filters from the Q- 
prediction filters. This allows us to deduce the prediction filters for the fourth level and for higher levels without relying 
on any calculations. 

[0076] Fig. 16 shows the implementation of a coder/decoder which can be used with the present invention imple- 
mented using a microprocessor 230 such as a Pentium IV from Intel Corp. USA. The microprocessor 230 may have 
an optional element such as a co-processor 224, e.g. for arithmetic operations or microprocessor 230-224 may be a 
bit-sliced processor. A RAM memory 222 may be provided, e.g. DRAM. Various I/O (input/output) interfaces 225, 226, 
227 may be provided, e.g. UART, USB, l 2 C bus interface as well as an I/O selector 228. FIFO buffers 232 may be used 
to decouple the processor 230 from data transfer through these interfaces. A keyboard and mouse interface 234 will 
usually be provided as well as a visual display unit interface 236. Access to an external memory such as a disk drive 
may be provided via an external bus interface 238 with address, data and control busses. The various blocks of the 
circuit are linked by suitable busses 231 . The interface to the channel is provided by block 242 which can handle the 
encoded video frames as well as transmitting to and receiving from the channel. Encoded data received by block 242 
is passed to the processor 230 for processing. 

[0077] Alternatively, this circuit may be constructed as a VLSI chip around an embedded microprocessor 230 such 
as an ARM7TDMI core designed by ARM Ltd., UK which may be synthesized onto a single chip with the other com- 
ponents shown. A zero wait state SRAM memory 222 may be provided on-chip as well as a cache memory 224. Various 
I/O (Input/output) interfaces 225, 226, 227 may be provided, e.g. UART, USB, l 2 C bus interface as well as an I/O 
selector 228. Fl FO buffers 232 may be used to decouple the processor 230 from data transfer through these interfaces. 
A counter/timer block 234 may be provided as well as an interrupt controller 236. Access to an external memory may 
be provided an external bus interface 238 with address, data and control busses. The various blocks of the circuit are 
linked by suitable busses 231. The interface to the channel is provided by block 242 which can handle the encoded 
video frames as well as transmitting to and receiving from the channel. Encoded data received by block 242 is passed 
to the processor 230 for processing. 

[0078] Software programs may be stored in an internal ROM (read only memory) 246 which may include software 
programs for carrying out subband decoding and/or encoding in accordance with any of the methods of the present 
invention. In particular software programs may be provided for digital filters according to embodiments of the present 
invention described above to be applied to a reference or other frame of data to generate one or more subbands of a 
set of subbands of an overcomplete representation of the frame by calculations at single rate. That is the software, 
when executed on the processor 230 carries out the function of any of the modules 22, 1 22, 48, 1 48, 1 28, 1 42 described 
above. Software may also combine the functions of several modules, e.g. the module 30 and module 22 of Fig. 1 a or 
the module 130 and module 122 of Fig. 9, or a combination of the modules 22, 28 and 30 of Fig. 1a or modules 122, 
128, 130 of Figs. 8, 9 or 11 or modules 122 and 128 of Fig. 8 or modules 128 and 130 of Fig. 11 or modules 142 and 
148 of Fig. 10. The methods described above may be written as computer programs in a suitable computer language 
such as C and then compiled for the specific processor in the design. For example, for the embedded ARM core VLSI 
described above the software may be written in C and then compiled using the ARM C compiler and the ARM assembler. 
Reference is made to "ARM System-on-chip", S. Furber, Addison-Wiley, 2000. The present invention also includes a 
data carrier on which is stored executable code segments, which when executed on a processor such as 230 will 
execute any of the methods of the present invention, in particular will execute digital filtering according to embodiments 
of the present invention described above to be applied to a reference or other frame of data to generate one or more 
subbands of a set of subbands of an overcomplete representation of the frame by calculations at single rate. The data 
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carrier may be any suitable data carrier such as diskettes ("f loopy disks"), optical storage media such as CD-ROMs, 
DVD ftOM's, tape drives, hard drives, etc. which are computer readable. 

[0079] Fig. 1 7 shows the implementation of a coder/decoder which can be used with the present invention imple- 
mented using an dedicated filter module. Reference numbers in Fig. 1 7 which are the same as the reference numbers 
5 in Fig". 1 6 refer to the same components - both in the microprocessor and the embedded core embodiments. 

[0080] Only the major differences will be described with respect to Fig. 1 7. Instead of the microprocessor 230 carrying 
out methods required to subband encode and decode a bitstream this work is now taken over by a subband coding 
module 240. Module 240 may be constructed as an accelerator card for insertion in a personal computer. The subband 
module has means for carrying out subband decoding and/or encoding in accordance with any of the methods of the 

10 present invention. In particular, the module may be provided with means for digital filtering according to embodiments 
of the present invention described above to be applied to a reference or other frame of data to generate one or more 
subbands of a set of subbands of an overcomplete representation of the frame by calculations at single rate. These 
filters may be implemented as a separate filter module 241 , e.g. an ASIC (Application Specific Integrated Circuit) or 
an FPGA (Field Programmable Gate Array) having means for digital filtering according to embodiments of the present 

is invention described above to be applied to a reference or other frame of data to generate one or more subbands of a 
set of subbands of an overcomplete representation of the frame by calculations at single rate. 
[0081 ] Similarly, if an embedded core is used such as an ARM processor core or an FPGA, a subband coding module 
240 may be used which maybe constructed as a separate module in a multi-chip module (MCM), for example or 
combined with the other elements of the circuit on a VLSI. The subband module 240 has means for carrying out subband 

20 decoding and/or encoding in accordance with any of the methods of the present invention. In particular, the module 
may be provided with means for digital filtering according to embodiments of the present invention described above 
to be applied to a reference or other frame of data to generate one or more subbands of a set of subbands of an 
overcomplete representation of the frame by calculations at single rate. As above, these filters may be implemented 
as a separate filter module 241 , e.g. an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable 

25 Gate Array) having means for digital filtering according to embodiments of the present invention described above. The 
present invention also includes other integrated circuits such as ASIC's or FPGA's which carry out the function of any 
of the modules 22, 122, 48, 148, 128, 142 described above. Such integrated circuits may also combine the functions 
of several modules, e.g. the module 30 and module 22 of Fig. 1a or the module 130 and module 122 of Fig. 9, or a 
combination of the modules 22, 28 and 30 of Fig. 1a or modules 122, 128, 130 of Figs. 8, 9 or 11 or modules 122 and 

30 1 28 of Fig. 8 or modules 128 and 130 of Fig. 11 or modules 142 and 148 of Fig. 10. 

[0082] While the invention has been shown and described with reference to preferred embodiments, it will be un- 
derstood by those skilled in the art that various changes or modifications in form and detail may be made without 
departing from the scope and spirit of this invention. 

35 TABLES 
[0083] 



Table I. 
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; The 9/7 filter-bank modified so that Dei = HqG^GqH^ 
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Table II. 



The prediction filters of level 1 for the 9/7 filter pair of Table I. 


Degree in z 
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Table ill. 



The prediction filters of level 2 for the 9/7 filter pair of Table L ] 


Degree In Z 
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Table III. (continued) 



The prediction filters of level 2 for the 9/7 filter pair of Table I. 


Degree In Z 
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Table IV. The prediction filters of level 3 for the 9/7 filter pair of Table I. 
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Table V. The multiplications required for the production of the subbands of level Jt 
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Table VI. Multiplication budget for the level-by-level production of the 
overcomplete representation. Two modes of operation are presented: FO-mode and 
HFO-mode. The results are presented per decomposition level both for the original 
and thresholdcd filters. Note: In notation F (F) 9 (F) is the number of filter-taps of 
filter F, with magnitude higher than the thresholds shown in Table VIII for the 
corresponding level /. 



Table VII. 

Comparison of the prediction-filters method with the LL-LBS method with respect to latency, delay and number 

of filter-kernels (F-Ks) used. A typical case of a 512-sample signal decomposed with the 9/7 filter-pair is presented. 

The numbers in parenthesis next to the filter-kernels show the maximum number of taps of these units (taking into 

account the filter symmetries) for every decomposition level. 
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Table VIII. 



The maximum mean-square error (MMSE) per decomposition level between the results of the original and the 
thresholded prediction-filters for the grayscale images of the JPEG-2000 test-set. The filters are applied in the row- 
column manner and the thresholds refer to the filters of the update-structure implementation of every level. 
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[0084] Table IX. Average PSNR for the first 48 frames of the football sequence at 30 frames per second. 
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Claims - 

1. A method of digital encoding or decoding a digital bit stream, the bit stream comprising a representation of a 
sequence of n-dimensional data structures, the method being of the type which derives at least one further subband 
of an overcompiete representation from a complete subband transform of the data structures, the method com- 
prising: 

providing a set of one or more critically subsampled subbands forming a transform of one data structure of 
the sequence; 

applying at least one digital filter to at least a part of the set of critically subsampled subbands of the data 
structure to generate a further set of one or more further subbands of a set of subbands of an overcompiete 
representation of the data structure, wherein the digital filtering step includes calculating at least a further 
subband of the overcompiete set of subbands at single rate. 

2. The method according to claim 1 , wherein a part of the data of one data structure of the sequence is mapped 
within predefined similarity critreria to a part of the data of a another data structure of the sequence. ' 

3. The method according to claim 1 or 2, wherein the digital filter is only applied to members of the set of critically 
subsampled subbands of the transform of the data structure. 

4. The method according to any previous claim, wherein the digital filter has at least two non-zero values. 

5. The method according to any previous claim, wherein the bit stream is a video bit stream. 

6. The method according to any previous claim, wherein the digital subband transform is a wavelet. 

7. The method according to any previous claim, wherein the data structures are data frames and the set of critically 
subsampled subbands of the transform of the data structure define a reference frame. 

8. The method according to any previous claim, wherein the subband transform is multilevel and the generation of 
the further set of one or more further subbands of a set of subbands of an overcompiete representation of the data 
structure is done level -by- level. 

9. The method according to any previous claim, wherein the method is block-based and the part of the data of one 
data structure is one block. 

10. Coderf or digital subband coding of a bit stream, the bit comprising a representation of a sequence of n-dimensional 
data structures, the coder being of the type which derives at least one further subband of an overcompiete repre- 
sentation from a complete subband transform of the data structures, the coder comprising: 

means for providing a set of one or more critically subsampled subbands forming a transform of one data 
structure of the sequence; 

means for applying at least one digital filter to at least a part of the set of critically subsampled subbands of 
the data structure to generate a further set of one or more further subbands of a set of subbands of an over- 
complete representation of the data structure, wherein the means for applying at least one digital filter includes 
means for calculating at least a further subband of the overcompiete set of subbands at single rate. 

11. The coder according to claim 10, wherein the coder is one of an encoder and a decoder. 
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12. The coder according to claim 10 or 11 , further comprising means to map a part of the data of one data structure 
oft he sequence to a part of the data of a another data structure of the sequence within predefined similarity critreria. 

13. The coder according to any of claims 10 to 12, wherein means for applying the digital filter applies the filter only 
tc members of the set of critically subsampled subbands of the transform of the data structure. 

14. The coder according to any of claims 10 to 1 3, wherein the data structures are data frames and the set of critically 
subsampled subbands of the transform of the data structure define a reference frame. 

15. The coder according to any of claims 1 0 to 1 4, wherein the coder is a decoder and data structures are data frames 
and the set of critically subsampled subbands of the transform of the data structure define a reference frame, 
further comprising: 

means to map a part of the data of one data structure of the sequence to a part of the data of a another data 
structure of the sequence within predefined similarity critreria and to generate a motion vector for that part 
and means to select a further subband of the overcomplete set of subbands in accordance with the motion 
vector. 

16. The decoder according to claim 15, further comprising means to provide a motion compensated representation of 
the reference frame using the selected further subband of the overcomplete set of subbands. 

17. The coder according to any of claims 10 to 16, wherein the subband transform is multilevel and the means to 
generate a further set of one or more further subbands of a set of subbands of an overcomplete representation of 
the data structure is done level-by-level. 

18. A computer program product comprising executable machine readable computer code which executes at least 
one digital filter for application to at least a part of a set of critically subsampled subbands of a data structure to 
generate a further set of one or more further subbands of a set of subbands of an overcomplete representation of 
the data structure, wherein the application of the at least one digital filter includes calculating at least a further 
subband of the overcomplete set of subbands at single rate. 

19. A data carrier storing the computer program product of claim 18. 

20. A digital filter module comprising means for application of a digital filter to at least a part of a set of critically 
subsampled subbands of a data structure to generate a further set of one or more further subbands of a set of 
subbands of an overcomplete representation of the data structure, wherein the application of the at least one digital 
filter includes calculating at least a further subband of the overcomplete set of subbands at single rate. 
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